perm filename JAN76.IN[LET,JMC] blob
sn#204197 filedate 1976-02-26 generic text, type C, neo UTF8
COMMENT ⊗ VALID 00194 PAGES
C REC PAGE DESCRIPTION
C00001 00001
C00020 00002 ∂03-JAN-76 0215 TAG
C00022 00003 ∂05-JAN-76 1201 BG Seminar on axiomatization of knowledge and time
C00024 00004 ∂06-JAN-76 0200 REM via ML UTILI.?[F75,JMC] remarks
C00028 00005 ∂06-JAN-76 1643 LES LISP70
C00029 00006 ∂06-JAN-76 2126 REM via AMET LISP.INT[F75,JMC]
C00031 00007 ∂06-JAN-76 2134 REM via AMET CRUNCH AND SPINDLE
C00033 00008 ∂07-JAN-76 0257 REM via AMET CRUNCH/SPINDL
C00034 00009 ∂07-JAN-76 0649 LES Gordon Travel Expense
C00035 00010 ∂08-JAN-76 1400 REG
C00037 00011 ∂08-JAN-76 2305 LES How would you like to visit CIA?
C00041 00012 ∂08-JAN-76 2327 LES
C00043 00013 ∂09-JAN-76 0009 LES
C00061 00014 ∂09-JAN-76 0152 LES Card Shuffling
C00064 00015 ∂09-JAN-76 1133 ZM Lisp,Pacini, and Vuillemin
C00066 00016 ∂09-JAN-76 1422 FXB schwartz
C00067 00017 ∂09-JAN-76 2000 LES Maybe we won
C00068 00018 ∂11-JAN-76 1657 CG
C00069 00019 ∂11-JAN-76 2030 ME Hacking mail
C00071 00020 I forsee no problem unless your recommendations are
C00074 00021 ∂12-JAN-76 1252 ZM optimal fixedpoints
C00077 00022 ∂12-JAN-76 1459 TW russel/carlstrom visit
C00078 00023 ∂12-JAN-76 1827 FTP:Geoff at SRI-AI SCCS mtg. in LA this coming weekend.
C00082 00024 ∂12-JAN-76 1956 LES Manna Proposal
C00083 00025 ∂12-JAN-76 2316 LES Fiction?
C00084 00026 ∂13-JAN-76 0934 LES
C00087 00027 ∂14-JAN-76 0125 LES Graphical Language paper
C00088 00028 ∂14-JAN-76 0953 FTP:Waldinger at SRI-AI zohar as principal investigator
C00093 00029 ∂15-JAN-76 1357 REM DISK ALLOCATION, REFORMATTING, SPINDLE...
C00098 00030 ∂15-JAN-76 1659 REM
C00099 00031 ∂15-JAN-76 2218 ZM REVAL - conclusion
C00100 00032 Patte, pleae check into this:
C00101 00033 ∂16-JAN-76 0952 LES III Negotiation
C00102 00034 ∂16-JAN-76 1108 RAK Refugee jobs
C00105 00035 ∂16-JAN-76 1620 FTP:CARL at MIT-AI
C00106 00036 ∂17-JAN-76 2244 REM via AMET PLANS (PRELIMINARY) FOR CRUNCH/SPINDLE
C00107 00037 I don't know any techniques, but I read that people devise algorithms
C00108 00038 ∂18-JAN-76 1301 100 : REM RE CONCORDANCES AND CORD-COUNTS
C00110 00039 ∂18-JAN-76 1556 REM via AMET PROCESS OF COUNTING TOKENS TO DECIDE ON CODE
C00112 00040 ∂18-JAN-76 1854 SGK Usage
C00113 00041 ∂18-JAN-76 1952 ME Creating files while in E
C00114 00042 ∂19-JAN-76 1125 BG John McCarthy's seminar
C00115 00043 ∂19-JAN-76 1138 REG
C00116 00044 ∂19-JAN-76 2324 JJK
C00117 00045 ∂20-JAN-76 0655 REM SO-FAR -- CRUNCHING PROJECT
C00119 00046 ∂20-JAN-76 1307 RF
C00121 00047 ∂20-JAN-76 2102 100 : Roode DECsystem-20
C00123 00048 ∂20-JAN-76 2203 DEW proposed chess project
C00124 00049 ∂21-JAN-76 0402 REM via AMET CRUNCH, PROGRESS REPORT
C00128 00050 ∂21-JAN-76 0421 REM via AMET RESULTS ON UUO.ME
C00129 00051 ∂21-JAN-76 0811 TAG
C00131 00052 ∂21-JAN-76 1100 JMC*
C00132 00053 ∂21-JAN-76 1340 SGK
C00133 00054 ∂21-JAN-76 1536 100 : REM via AMET HOTER.ESS MENTAL
C00136 00055 ∂21-JAN-76 2015 RS via IMSSSS Your manuscript "Ascribing mental qualities to machines"
C00138 00056 ∂21-JAN-76 1106 JMC
C00139 00057 ∂22-JAN-76 0818 REM RESULTS (DIGRAMS AND TRIGRAMS) SO FAR
C00140 00058 ∂22-JAN-76 0910 LES Russell-Carlstrom Visit Rescheduled
C00142 00059 ∂22-JAN-76 0937 REM
C00143 00060 ∂22-JAN-76 1023 REM 3-LETTER STRINGS
C00144 00061 ∂22-JAN-76 1435 BG
C00145 00062 ∂22-JAN-76 1508 100 : queenie COMPUTER FORMUM
C00147 00063 ∂22-JAN-76 2253 100 : REM via AMET Reply to your question.
C00149 00064 ∂22-JAN-76 2311 FTP:REM at MIT-ML
C00150 00065 ∂22-JAN-76 2328 TAG
C00151 00066 ∂22-JAN-76 2339 TAG
C00153 00067 ∂23-JAN-76 0241 ME Imlac problem
C00154 00068 ∂23-JAN-76 0338 FTP:REM at MIT-ML
C00155 00069 ∂23-JAN-76 0927 LES FEEPME
C00156 00070 ∂23-JAN-76 1026 LES ARPA games
C00158 00071 ∂23-JAN-76 1243 QIB RICHARD WEYHRACUH
C00159 00072 ∂23-JAN-76 1635 REM via AMET Results of library search
C00162 00073 ∂23-JAN-76 1649 REM via AMET
C00163 00074 ∂24-JAN-76 1525 LES The next operating system
C00167 00075 ∂24-JAN-76 1929 RWW ARITHMETIC
C00168 00076 ∂25-JAN-76 1707 REF via CMUA
C00169 00077 ∂25-JAN-76 1745 LES Label Printer
C00170 00078 Seem unlikely to me.
C00171 00079 ∂25-JAN-76 2026 LES Manna PI memo
C00172 00080 ∂26-JAN-76 1139 JH SENSE
C00174 00081 ∂26-JAN-76 1908 FTP:CERF at USC-ISI (Response to message)
C00175 00082 ∂27-JAN-76 0701 REM FIRST ESTIMATE OF CRUNCHING COMPRESSION RATIO
C00176 00083 ∂27-JAN-76 0735 REM MORE FIRST-RESULTS
C00177 00084 ∂27-JAN-76 1508 100 : REM via AMET Reply -- Files containing info on parse...
C00179 00085 ∂27-JAN-76 1546 FTP:REM at MIT-ML
C00183 00086 ∂27-JAN-76 1917 REM Reply about "qu" and "the " in HOTER.ESS
C00184 00087 ∂28-JAN-76 0148 FTP:Omalley at SRI-AI Next BAAIC meeting on 2/18
C00186 00088 ∂28-JAN-76 0326 REM THE DETAILS OF THE HUFFMAN CODE YOU REQUESTED.
C00187 00089 ∂28-JAN-76 0412 ME
C00188 00090 ∂28-JAN-76 0528 REG
C00192 00091 ∂28-JAN-76 0547 REM More of same for comparison
C00193 00092 ∂27-JAN-76 1305 JMC
C00194 00093 ∂28-JAN-76 0554 REM ONE MORE THING TO TRY
C00195 00094 ∂28-JAN-76 2000 REM via AMET History-tree Crunching
C00200 00095 ∂29-JAN-76 0100 ME
C00202 00096 ∂29-JAN-76 0336 RHT
C00203 00097 ∂29-JAN-76 1208 LES IMLAC & air conditioning
C00206 00098 ∂29-JAN-76 1233 LES Kant phone
C00207 00099 ∂29-JAN-76 1426 RWW
C00208 00100 ∂29-JAN-76 1500 LES Oppen reappointment
C00210 00101 ∂29-JAN-76 1636 LES Subroutine library
C00211 00102 ∂29-JAN-76 2248 LES Edit & mail to processes
C00214 00103 ∂30-JAN-76 0256 REM CORRECTION, MINOR
C00217 00104 ∂30-JAN-76 1948 100 : Sarah Overnight whereabouts
C00219 00105 ∂30-JAN-76 2035 REM Number base-conversion, emitting "less than one bit"
C00222 00106 ∂30-JAN-76 2051 REM Continuation...
C00225 00107 ∂30-JAN-76 2232 JJK Equipment for Computer Forum
C00226 00108 ∂31-JAN-76 0617 REM HISTORY (LEFT-CONTEXT) SINGLE-CHARACTER HUFFMAN-CODE
C00229 00109 ∂31-JAN-76 1646 REM via AMET
C00231 00110 ∂01-FEB-76 1721 LES Jeff Rosen Account
C00232 00111 ∂01-FEB-76 1749 FTP:HQM at MIT-AI
C00234 00112 ∂01-FEB-76 2033 FTP:HQM at MIT-AI
C00235 00113 ∂01-FEB-76 2035 BPM <MCCARTHY>@USC-ISI
C00236 00114 ∂02-FEB-76 0349 REM via AMET C.R. compared with results of J.P.McCarthy
C00239 00115 ∂02-FEB-76 2238 FTP:KING at PARC-MAXC Computer Forum
C00241 00116 ∂03-FEB-76 0544 REM MORE RESULTS, HISTORY-HUFFMAN, AND LIBRARY
C00246 00117 ∂03-FEB-76 1026 HVA Computer Forum Events
C00247 00118 ∂03-FEB-76 1039 HVA CORRECTION -LUNCHES - Computer Forum
C00248 00119 ∂03-FEB-76 1149 ZM
C00249 00120 ∂03-FEB-76 1919 LES BBN Pager is Safe (I think)
C00250 00121 ∂03-FEB-76 2012 REM via AMET "WORST-POSSIBLE" RESULTS ARE NOT TOO BAD.
C00253 00122 ∂05-FEB-76 0529 REM Listing files of results so far
C00254 00123 ∂05-FEB-76 1652 REM via AMET Those listing files
C00256 00124 ∂05-FEB-76 1751 REM via AMET USING HH3A.LST (I THINK) AS INPUT TO CRUNCHER
C00257 00125 ∂05-FEB-76 1908 REM via AMET ADDENDA TO PREVIOUS NOTE
C00258 00126 ∂06-FEB-76 0002 REM NEW RESULTS, <=50% C.R. OVERALL
C00263 00127 ∂06-FEB-76 1213 FTP:REM at MIT-ML
C00264 00128 ∂06-FEB-76 1551 PLW ai lab account
C00265 00129 ∂06-FEB-76 1646 PAW letter to Leon Knopoff
C00266 00130 ∂06-FEB-76 1650 PAW documentation
C00267 00131 ∂06-FEB-76 0946 JMC crunch
C00269 00132 ∂08-FEB-76 2109 KRD an amusing afternoon...
C00271 00133 ∂09-FEB-76 0653 REM GRIMM[LIB,DOC]
C00272 00134 ∂09-FEB-76 0722 REM WUTHER[LIB,DOC]
C00273 00135 ∂09-FEB-76 1937 WD National Engine Removal Week
C00275 00136 ∂09-FEB-76 2246 100 : REM via AMET Wuther, more left-context, ...
C00277 00137 ∂10-FEB-76 0640 FTP:MASINTER at PARC-MAXC file understanding system
C00278 00138 ∂10-FEB-76 1210 ZM Aharon Gill
C00279 00139 ∂10-FEB-76 1631 100 : REM Bill Kautz
C00282 00140 ∂10-FEB-76 1810 REM Crunch, purge, ...
C00283 00141 ∂10-FEB-76 1829 REM System support for large use of crunching
C00286 00142 ∂11-FEB-76 0605 REM via AMET SO FAR
C00288 00143 ∂11-FEB-76 1215 LES ONR Proposal
C00289 00144 ∂12-FEB-76 0328 REM via AMET Working crunch and uncrunch
C00291 00145 ∂12-FEB-76 0334 REM via AMET
C00292 00146 ∂12-FEB-76 0543 REM via AMET
C00293 00147 ∂12-FEB-76 0551 REM via AMET Using a code to crunch other files not surveyed, first result.
C00295 00148 ∂12-FEB-76 1946 HPM BLACK WATCH
C00296 00149 ∂12-FEB-76 2023 REM via AMET SNEKOT.DAT
C00298 00150 ∂12-FEB-76 2049 REM via AMET
C00299 00151 ∂13-FEB-76 0035 REM via AMET WUTHER, SNEKOT, CRU1, SYSTEM
C00303 00152 ∂13-FEB-76 1202 REM WUTHER WITH AUGMENTED LEFT CONTEXTS
C00305 00153 Here is a message from Cerf that refers to something you said:
C00310 00154 ∂15-FEB-76 0013 KRD oral exam date, finally
C00311 00155 ∂15-FEB-76 1838 CG WISE MAN PROBLEM
C00312 00156 ∂15-FEB-76 2101 REM CRU1 AND CRU2
C00314 00157 ∂16-FEB-76 0428 REM TIMING DATA ON CRU2
C00317 00158 ∂16-FEB-76 0845 REM TIMING RESULTS WITH FAST-CODE IMPLEMENTED ALSO
C00319 00159 ∂16-FEB-76 0854 REM FASTER THAN COPY!! MORE POWERFUL THAN A LOCOMOTIVE.
C00322 00160 ∂16-FEB-76 1059 BPM MAIL PAT%IMSSS
C00323 00161 ∂16-FEB-76 1538 REM VERSION-NUMBER IDENT
C00325 00162 ∂16-FEB-76 1557 REM CONTINUATION OF LAST MESSAGE...
C00328 00163 ∂17-FEB-76 0224 REM
C00329 00164 ∂17-FEB-76 0759 REM via AMET
C00330 00165 ∂17-FEB-76 1409 FXB 11/70
C00331 00166 ∂17-FEB-76 1814 HQM imlac graphics
C00332 00167 ∂18-FEB-76 0106 CG WISE MAN PROBLEM
C00333 00168 ∂18-FEB-76 0212 DCL Gerard Huet summer visit
C00337 00169 ∂19-FEB-76 0124 REM IMPROVED CRU2.DMP[1,REM]
C00339 00170 ∂19-FEB-76 0136 REM PTYJOB
C00340 00171 ∂19-FEB-76 0142 REM TTY11 PERSISTANT TROUBLE
C00342 00172 ∂19-FEB-76 0545 REM via AMET
C00343 00173 ∂19-FEB-76 0600 REM via AMET IMPROVEMENTS TO CRU2
C00345 00174 ∂19-FEB-76 1205 REG
C00346 00175 ∂20-FEB-76 0458 REM via AMET New features in CRU2
C00347 00176 ∂20-FEB-76 0505 REM at TTY122 0505 via AMET
C00348 00177 ∂20-FEB-76 1109 FTP:PHW at MIT-AI
C00349 00178 ∂20-FEB-76 1430 REM
C00350 00179 ∂20-FEB-76 1721 FTP:Dehall at SRI-AI Berkeley Workshop on Data Management and Computer Networks.
C00356 00180 ∂20-FEB-76 1416 JMC BUG
C00359 00181 ∂21-FEB-76 1635 REM via AMET Improvements in CRU2
C00361 00182 ∂22-FEB-76 1225 REG
C00362 00183 ∂22-FEB-76 1514 100 : ED FREDKIN via AI LOGGING IN TO SU-AI
C00363 00184 ∂23-FEB-76 1552 LES Jack Buchanan call
C00364 00185 ∂23-FEB-76 1622 LES
C00366 00186 ∂23-FEB-76 1802 EF via AI login
C00367 00187 ∂24-FEB-76 0049 LES Research Assistantship
C00368 00188 ∂24-FEB-76 0157 KRD reminder
C00369 00189 ∂24-FEB-76 0939 TED tty11
C00370 00190 ∂24-FEB-76 1006 FTP:MOORE at MIT-AI
C00372 00191 ∂24-FEB-76 1400 JMC*
C00379 00192 ∂25-FEB-76 0925 LES III licensing deal
C00380 00193 ∂25-FEB-76 1539 RWW ACCOMPLISHMENTS
C00381 00194 ∂25-FEB-76 1913 REM via AMET SPINDLE FORMAT ETC.
C00383 ENDMK
C⊗;
∂03-JAN-76 0215 TAG
I TALKED TO BO FRIDAY EVENING, AND HE SAYS THAT NO IMLAC DIAGNOSTIC
SOFTWARE WAS EVER MADE TO WORK ON THE MODIFIED IMLACS. THE ASSEMBLY SWITCH
THAT QUAM IS TALKING ABOUT IS PROBABLY THE SWITCH IN THE IMLAC OPERATING
PROGRAM WHICH ASSEMBLES SPECIAL DISPLAY CODE FOR THE EX-QUAM IMLAC, WHICH
HAS BEEN MODIFIED DIFFERENTLY FROM THE OTHER TWO IMLACS. WE DO NOT POSESS
SOURCES FOR IMLAC DIAGNOSTICS. HOWEVER, BO HAS VOLUNTEERED TO HACK AN IMLAC
DISASSEMBLER SOME EVENING AT THE LAB, A PROJECT WHICH HE SAYS WILL NOT TAKE
VERY LONG, AS HE DID ONE IN SAIL FOR A NOVA SOME TIME AGO, WHICH ONLY TOOK
3 HOURS. WITH A DEXX DISASSEMBLER, WE CAN MODIFY IMLAC DIAGNOSTICS TO RUN
ON OUR IMLACS, AND THEREBY INCREASE THE CHANCES FOR GETTING THE DEAD ONE
FIXED.
∂05-JAN-76 1201 BG Seminar on axiomatization of knowledge and time
To: PEOPLE.SEM[DOC,BG]:;
The seminar will meet at 4:00 p.m. today [Monday, January 5] in the conference
room of the a.i. laboratory. Bob Moore will speak; I think the subject will
be his MIT master's thesis "Reasoning from Incomplete Knowledge in a Procedural
Deduction System."
∂06-JAN-76 0200 REM via ML UTILI.?[F75,JMC] remarks
To: LES, JMC
My first impression is that you are looking for a combination of the
cruncher (entropy-based data-coding) and the spindler (also called
an archiver on the Harvard system I believe, in effect reformatting
a portion of the disk to implement smaller chunks than the 2.3K we
are currently using as the quantum of disk space).
∂06-JAN-76 1643 LES LISP70
Dave Smith has requested that his allocation be increased to 300k in order
to carry on with LISP70. This raises a question in my mind about the
extent of our commitment to LISP70. Perhaps you would like to question
DAV.
∂06-JAN-76 2126 REM via AMET LISP.INT[F75,JMC]
LISP (INTER- and UCI-) still exist because people (incl. myself)
use it for some research applications. We use it because we don't know
of any better programming system for those types of applications.
We like it because it has a mathematical beauty (data and program are the
same and inter-usable, evaluation is recursive for both program and data
in about the same way, EVAL is particularily nice in the way it
processes all expressions in the same way) and simplicity (the CAR is
always the function, no wierd syntaxes, all things can be handled by
the same mechanism using various functions with different names, editing
according to structure is straightforeward and logical).
∂06-JAN-76 2134 REM via AMET CRUNCH AND SPINDLE
I assume you want a single program to do the duties formerly
done by both the spindler/archiver (SPINDL which reformats the disk
at the user level, allowing files to start at places other than
IBM 3330 block 2.3k boundaries) and the cruncher/coder (JMCWC1 which
encode data according to entropic considerations so that it uses
a minumum number of bits). Please verify.
Assuming you verify, there are a large number of ideas I've had
over the years as to how to compress ascii files although I've not yet
implemented any particular system because I've not yet been satisfied
completely with any. Perhaps we can implement a full archiver with
a rather dumb/easy/simple cruncher, then upgrade the cruncher later
if and when we find a significant improvement in compression ratio with
only moderate increase in compute time.
∂07-JAN-76 0257 REM via AMET CRUNCH/SPINDL
CRUNCH.PLN[1,REM] contains some remarks of mine to tide you over
until I call.
∂07-JAN-76 0649 LES Gordon Travel Expense
Based on a message that I just received from Al Blue, it is unlikely that
we will get Surra unwedged on the Michael Gordon travel issue. Based
on our letter offer to him, we have a commitment to pay the return fare.
Would you accept the honor?
∂08-JAN-76 1400 REG
[The following message to you got lost in the mail - REG]
∂05-JAN-76 1143 ZM OEVAL vs. REVAL
both OEVAL and REVAL seems to me to be correct.
REVAL will yield in certain cases less SUBSTITUTIONS ( as Pacini claims),
but it does not mean that the algorithm is more efficient.
For example, for FIB(4)
OEVAL needed 8 substitutions and 103 ENTER OEVAL
while
REVAL needed 8 subs. and 118 ENTER REVAL.
I try to compare Pacini's algorithm with Vuillemin's algorithm.
Compute please F(2,1) with OEVAL and REVAL where
F(x,y) ← if x=0 then y+1
else if y=0 then F(x-1,1)
else F(x-1,F(x,y-1)). Zohar
∂08-JAN-76 2305 LES How would you like to visit CIA?
To: ME
CC: JMC
Martin:
Here's a message I just received. We can talk about it later.
∂08-JAN-76 1143 FTP:ICE at USC-ISI MARTIN FROST VISIT--POSSIBLE?
Date: 8 JAN 1976 1140-PST
From: ICE at USC-ISI
Subject: MARTIN FROST VISIT--POSSIBLE?
To: EARNEST at SU-AI
cc: WALKER
Les:
Thank you for so patiently answering my previous list of question
about News Service, but you're not off the hook yet. We finally
have our various offices that are interested in NS coordinated,
and the unanimous decision is tht, if Martin Frost is willing, we
would like to have him visit us for a few days and review our
problems. I am willing to correspond with him directly if you
are tired of being the middle man. Until then, could you ask him
to let us know*
(1) What would be a convenient time to visit DC?
(2) What would the approximate costs be--travel, hotel, consul-
ting fees, etc.?
(3) Does he have any security clearances and if so, what level,
from which agency (DOD, NSA, etc.), and what is his social
security number (not necessary if he has no clearances).
We have a few other questions about NS which either you or he might be
able to answer:
(1) How much disk space is used to store 2 weeks worth of the
2 wire services product?
(2) How much core is used to run the NS program?
(3) What, if any, are your future plans for the system. Had you
intended to ever offer it as a service or it is a research tool only?
I'm heading for New Hampshires' White Mountains weekend after this,
for some cross-country skiiing and winter camping. I've never tried
cross-country before--so wish me luck!
And do return to the Gaspe sometime--it is wonderful. You would also
like Gros Morne National Park (still largely undeveloped) in Newfound-
land.
Roxanne (ICE at ISI)
-------
∂08-JAN-76 2327 LES
On Wednesday, ARPA (in the form of Al Blue) finally discovered that
they gave us the KA-10 last year. The way this happened was that
when I requested approval to get the KL-10 electrical power
installed, he said that they would like to take back the KA-10
because they have a severe shortage of processors at the moment.
I politely mentioned the title transfer and subsequently provided
pointers to the documentation (we have had three message exchanges so
far). Now I have received the following note. He may be kidding,
but I don't think so. My inclination is to call him an "indian
giver", but seriously offer to give it back if he means it. Any
opinions?
------------
∂08-JAN-76 1230 FTP:BLUE at USC-ISI ARPA Indians
Date: 8 JAN 1976 1226-PST
From: BLUE at USC-ISI
Subject: ARPA Indians
To: les at SU-AI
cc: Blue
Les:
After a long file search, I have to admit that you seem to b4e correct!
With whom should I deal in regard to 'correcting' the equipment inventory
that was appended to the letter transferring title? You for the Univ.?
(I can handle it from this end, and I assume I can get Surra to go along.)
If you are the guy, call me or drop me a msg and I'll call you.
-------
∂09-JAN-76 0009 LES
Sorry for the delay. I managed to tie myself in a knot while copying
these messages.
∂07-JAN-76 0417 LES Computer accounts and other assorted topics
To: Blue @ USC-ISI
Al:
This is a catchall note: some followup on pending items and a couple
of new ones.
0. We were most pleased to hear that Heilmeier signed off on our
contract renewal. Since the old one has now lapsed and the Stanford
administration tends to get uptight in this situation, any information
that you can give on where the pipeline we are would be helpful
in keeping our creditors at bay.
1. Just before Christmas, I sent you a note requesting access to the
SRI machine for Dershowitz, which I hope is still in your queue.
2. Did you ever succeed in getting a clarification from Phil Surra
of the Michael Gordon travel issue? As you may recall, he seemed
to be saying one thing to your office and something else to us.
Meanwhile, many months later, Gordon still doen't have his air fare.
I'd like to close the books on this one: I'm willing to settle for
a statement that Surra has succeeded in doing it to us again. If
so, I will start looking for other sources of money to fulfill our
commitment.
3. We have been receiving some inquiries from commercial enterprises
that would like to use software that we have developed, largely
with ARPA support. Do you offhand know what constraints, if any,
there are on this, or can you point at an expert?
4. When you set up accounts at ISI for Barstow and Kant, you remarked
that they would likely move to the ISIC machine sometime soon.
Barstow would like to do it as soon as possible. Also, they both
request an increase in their disk allocations to 200 pages, since
they are both saturated and are shuffling files back and forth to
avoid being purged.
Our people who are running on the ISI machines often have questions
about services or have requests that probably don't need very high
level review. Is there someone at ISI who we could poke on routine
questions, such as the move to the C machine? My people claim to
have tried and were told that all such requests must be routed
through ARPA. I hate to bother you about such things. [Also, to be
honest, I don't like to be bothered myself.]
Thanks for your help.
Les
∂07-JAN-76 0630 FTP:BLUE at USC-ISI Re: Request for approval of KL10 electrical power procurement
Date: 7 JAN 1976 0629-PST
From: BLUE at USC-ISI
Subject: Re: Request for approval of KL10 electrical power procurement
To: LES at SU-AI
In response to your message sent 7 JAN 1976 0519-PST
1. DSSW has your ARPA Order package. Direct communication with
Bud Allen there is your best bet at this point.
2. Gordon travel: Surra appears to have won this. Believe it has to
be chalked up to the cost of doing oral business with Lick. We must,
in future, get a written record of such transactions.
3. Without looking, I expect your contract has a standard Rights In Data
clause which gives the Gov't non-exclusive rights to all software
produced, with commercial rights more or less in the public domain.
This is an area that must have been somewhat clarified since last
I became involved in it (it certainly was muddy then). Suggest you
begin by asking your Contracting Officer (Bud Allen again) for a
ruling.
4. Sorry, I forgot about Dershowitz. Will fix it today. Same goes for
Barstow and Kant.
5. Questions of the type you describe should go to <action>, copy to <dale>,
at the system you are inquiring about. If it involves establishment,
transfer or increasing page allocations of users, it should come to me.
-------
!∂07-JAN-76 0519 LES Request for approval of KL10 electrical power procurement
To: Russell @ USC-ISI, Blue @ USC-ISI
Dave & Al:
This is an experiment. Below is a routine request for subcontract
approval. I can get these to you faster over the network than by
U.S. mail. My question is, do you prefer to receive the hard copy,
or do you want both, or is this all a bad idea?
Sincerely,
Les
------------------------------------------------------------------------
∂07-JAN-76 0630 FTP:BLUE at USC-ISI Re: Request for approval of KL10 electrical power procurement
Date: 7 JAN 1976 0629-PST
From: BLUE at USC-ISI
Subject: Re: Request for approval of KL10 electrical power procurement
To: LES at SU-AI
In response to your message sent 7 JAN 1976 0519-PST
1. DSSW has your ARPA Order package. Direct communication with
Bud Allen there is your best bet at this point.
2. Gordon travel: Surra appears to have won this. Believe it has to
be chalked up to the cost of doing oral business with Lick. We must,
in future, get a written record of such transactions.
3. Without looking, I expect your contract has a standard Rights In Data
clause which gives the Gov't non-exclusive rights to all software
produced, with commercial rights more or less in the public domain.
This is an area that must have been somewhat clarified since last
I became involved in it (it certainly was muddy then). Suggest you
begin by asking your Contracting Officer (Bud Allen again) for a
ruling.
4. Sorry, I forgot about Dershowitz. Will fix it today. Same goes for
Barstow and Kant.
5. Questions of the type you describe should go to <action>, copy to <dale>,
at the system you are inquiring about. If it involves establishment,
transfer or increasing page allocations of users, it should come to me.
-------
∂07-JAN-76 0519 LES Request for approval of KL10 electrical power procurement
To: Russell @ USC-ISI, Blue @ USC-ISI
Dave & Al:
This is an experiment. Below is a routine request for subcontract
approval. I can get these to you faster over the network than by
U.S. mail. My question is, do you prefer to receive the hard copy,
or do you want both, or is this all a bad idea?
Sincerely,
Les
------------------------------------------------------------------------
Stanford Artificial Intelligence Laboratory
Stanford University
Stanford, California 94305
Telephone 415 497-4971 7 January 1976
Mr. Philip Surra
Resident Representative, O.N.R.
Room 165, Durand Building
Stanford University
Stanford, California 94305
Subject: Electrical Power Requirements for New Computer (Contract
DAHC15-73-C-0435)
Dear Mr. Surra:
This is a request for approval to contract for installation of
electrical power facilities for a KL10 processor, which is to be used
in support of our ARPA-sponsored research program, at a cost of
$3,880. The University is unable to provide sufficient funds from
its own resources to procure this facility. There are sufficient
funds available under the subject contract to cover this cost.
As you may recall, Digital Equipment Corporation is planning to
donate to this Laboratory a KL10, worth something over $200,000. It
is currently slated to arrive in February, although a confirmed date
has not yet been received. This machine consumes a great deal more
electrical power than our existing computer, so new major circuits
must be brought in.
We have obtained and reviewed three bids on this task from local
groups and propose to contract with the least expensive (Mausser
Electric Co.). We request your approval of this undertaking.
Sincerely,
Lester D. Earnest
Associate Director
cc: Dave Russell (ARPA), Al Blue (ARPA).
∂07-JAN-76 0611 FTP:BLUE at USC-ISI Re: Request for approval of KL10 electrical power procurement
Date: 7 JAN 1976 0610-PST
From: BLUE at USC-ISI
Subject: Re: Request for approval of KL10 electrical power procurement
To: LES at SU-AI
In response to your message sent 7 JAN 1976 0519-PST
Les:
The idea of sending such info copies via net is a good one. Let us
assume that you will follow this practice in the future.
HOWEVER, the subject of the one you just sent is below IPTO threshhold.
We have no reason to chop on a 3.8K procurement whatsoever. Nor do we
approve sub-contracts of this magnitude. This is Surra's job.
In this regard, though, I will be surprised if he doesn't take the
position that such an expense should come from University overhead
inasmuch as the University owns the computer that it will primarily
serve.
←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←
P. S. - Col Russell wants to know the status of the KA-10 after the KL
is operational. Please send us a message on this. Be
warned that IPTO is KA-hunting; your justification for keeping
both the KA and KL must be good. Even so, the decision may go
against you.
-------
∂07-JAN-76 0624 LES KA-10 Plans
To: Blue @ USC-ISI
Al:
Thanks for quick response.
On the electrical power thing, it is unlikely that the University
would be willing to put up general funds in support of a computer
that is being used primarily in support of ARPA research and for
which no direct charges to the contract are being made.
On the KA-10, we had planned to retire our old PDP-6 and use the
KA-10 for its functions (mainly XGP and vision processes). Also, it
actually belongs to Stanford, the title having been transferred last
year, although I suppose that we could give it back.
∂07-JAN-76 0836 FTP:BLUE at USC-ISI Re: KA-10 Plans
Date: 7 JAN 1976 0835-PST
From: BLUE at USC-ISI
Subject: Re: KA-10 Plans
To: LES at SU-AI
In response to your message sent 7 JAN 1976 0641-PST
Hey, refresh my memory on giving you the KA...I recall we gave you title
to the 6 so you could swap it for the KL. But the ten?????
-------
∂08-JAN-76 0447 LES KA-10 title
To: Blue @ USC-ISI
Al:
Yes, you really seem to have given us the KA-10.
To recap the history, I discussed the capital equipment list with you
early in 1974 and you suggested that we submit a request for transfer
of title. This was long before the prospect of trading the PDP-6
arose. I sent a letter addressed to you in April 1974 [Ref. 1], that
listed essentially everything on the books.
Subsequently Phil Surra became upset over the fact that the request
was addressed to you, so we resubmitted it to him in March 1975
[Ref. 2].
In subsequent discussions, I was asked several times why we wanted
the transfer and I referred to the prospective PDP-6 trade-in. I got
the impression from those discussions that only the PDP-6 and some of
the other old and/or inexpensive stuff would be transferred.
Nevertheless, when the transfer came in [Ref. 3], it covered
everything on the list. How could I complain?
We certainly didn't intend to flim-flam anyone. If you want copies of
any of the correspondence, I can send it.
[1] L. Earnest, letter to A. Blue, 5 April 1975.
[2] J. Erickson (Stanford), "Transfer of Title to Equipment/Follow-up",
letter to P. Surra, 27 March 1975.
[3] P. Surra, letter to J. Erickson, 24 April 1975.
∂09-JAN-76 0152 LES Card Shuffling
To: Blue @ USC-ISI
CC: Russell @ USC-ISI, JMC
You know, we used to be called the "Stanford Indians", but gave that
up after ethnic protests. It's now the "Stanford Cardinals". I've
never heard of the "ARPA Indians". Is that a football team?
I am not certain of the propriety of a non-profit institution making
donations to the government. We have to be careful about our tax
status, you know. Now if we could sell it back to you ...
More seriously, we haven't thought through the implications of
possibly losing the KA-10 yet, but can identify a couple of potential
problems. If the PDP-6 goes as a trade-in on the KL10 and if the
KA10 went away, then we would have no peripheral processor to run the
XGP, among other things. It is not clear whether the KL10 could take
on that task and still perform adequately.
From your viewpoint, there is a problem in the processor
modifications that have been made over the years, some of them not
adequately documented. Turning it back into a virgin KA10 would give
someone a few gray hairs.
If you would like to talk about this, I'll still be here when you get
in in the morning. In fact, I plan to be here until about 1 PM your
time, when I stagger off to a meeting.
∂09-JAN-76 1133 ZM Lisp,Pacini, and Vuillemin
You are right - all 3 rules require 14 substitutions.
Here is an example that will clarify everything (I hope).
Consider F(2,1) where
F(x,y) ← if x=0 then 1 else F(x-1,F(x-y,y)).
Lisp will require 7 substitutions (marked by ↑):
F(2,1) → F(1,F1,1)) → F(1,F(0,F(0,1))) → F(1,F(0,1)) → F(1,1) →
↑ ↑ ↑ ↑ ↑
F(0,F(0,1)) → F(0,1) → 1
↑ ↑
Vuillemin`s rule will require only 3 substitutions:
F(2,1) → F(1,F(1,1)) → F(0,F(1-F(1,1),F(1,1))) → 1
↑ ↑ ↑
If I understand Pacini`s rule correctly, it will require 5 subst.:
F(2,1) → F(1,F(1,1)) → F(1,F(0,F(0,1))) → F(1,1) → F(0,F(0,1)) → 1.
↑ ↑ ↑ ↑ ↑
Please try to run this example.
∂09-JAN-76 1422 FXB schwartz
Since your phone is busy, I'm trying anoher input channel.
Will you meet Schwartz Monday at 4 at the lab and then
presumably worry about his evening meal?
-Forest
∂09-JAN-76 2000 LES Maybe we won
∂09-JAN-76 1352 FTP:BLUE at USC-ISI Re: Telephone call
Date: 9 JAN 1976 1350-PST
From: BLUE at USC-ISI
Subject: Re: Telephone call
To: LES at SU-AI
In response to your message sent 9 JAN 1976 0845-PST
Les: Sorry I kept you on the hook - or rather off it. Rest easy over the
weekend; I may have talked Russell out of it based on your claim of
imperfect documentation of cpu diddles.
-------
∂11-JAN-76 1657 CG
A write up of some of my ideas on knowledge are in KNO[1,CG]. I din't get
as far as I hoped to in the writing, but I will discuss further ideas
at the seminar tomorrow.
∂11-JAN-76 2030 ME Hacking mail
I'm not putting much effort at all into hacking mail. RWW asked for
a feature that seemed reasonable and which was very easy so implement
(about 1 man-hour) so I put it in. About LOGIN:NOMAIL being the
default, I would agree but it would force everyone who doesn't have
the NOMAIL option (and who doesn't want it) to put in some MAIL
option. The default isn't too important here; people can choose
either option. Actually, I was originally going to flush reading
mail from LOGIN altogether, thus forcing NOMAIL not only as the
default, but as the only option. In fact, I did that for about 3
hours a few weeks ago, but some people complained, so I undid it.
I forsee no problem unless your recommendations are
long delayed.
∂12-JAN-76 0805 FTP:RYLAND at USC-ISID Possibly Late Grad School Application
Date: 12 JAN 1976 0805-PST
Sender: RYLAND at USC-ISID
Subject: Possibly Late Grad School Application
From: RYLAND at USC-ISID
To: jmc at SAIL
Cc: RYLAND
Message-ID: <[USC-ISID]12-JAN-76 08:05:07-PST.RYLAND>
Special-Handling: Please read soon
Dear Professor McCarthy:
This is Chris Ryland at Harvard, one of Tom Cheatham's
students. I will be applying to the Stanford Computer
Science Department for graduate study, but I let things get
down to the wire with my application, and because I have
been in bed with the flu for the past couple of days, it
looks like I won't be able to mail my application until
tomorrow, the 13th. Although I will send it special
delivery, it very likely won't make it to you until the
16th, so I'm hoping that you will allow for this mixup. I
would hate to see my chances for admission disappear because
of a last-minute delay. I'm truly looking forward to
hearing from you eventually. Thank you for your trouble,
and
Cheers,
Chris Ryland, Harvard '76
(I can be reached via the Net at, in descending order of
reliability, Ryland@ISID, Ryland@Rutgers-10, Ryland@CMUA,
Ryland@Harv-10.)
-------
∂12-JAN-76 1252 ZM optimal fixedpoints
You are definitely rigth - optimal fixedpoints may be complex, etc.
(I never thought about this ! ).
It may be very interesting to try to characterize classes of
recursion where this happens.
I shall look further into this problem. Thanks Zohar
The idea that optimal fixed points might be complex, etc.
occurred to me as a consequence of an intuition that they don't
have much to do with computer science per se, but are rather a
development in functional equations in that the conditional
expression recursion permits integrating the main functional
equation with its boundary conditions.
∂12-JAN-76 1459 TW russel/carlstrom visit
To: JMC, LES
If at all possible, I would rather not have meetings scheduled that
I have to be at which conflict with my 9:30 --11:00 class.
We'll try.
∂12-JAN-76 1827 FTP:Geoff at SRI-AI SCCS mtg. in LA this coming weekend.
Date: 12 JAN 1976 1819-PST
From: Geoff at SRI-AI
Subject: SCCS mtg. in LA this coming weekend.
To: jmc at SAIL
12-JAN-76 16:53:20-PST,1201;000000000000
Mail from USC-ISID rcvd at 12-JAN-76 1653-PST
Date: 12 JAN 1976 1653-PST
From: CARLISLE at USC-ISID
Subject: Southern California Computer Society
To: bboard at ISIB
cc: moore, levin, richardson, mann
SCCS
The next meeting of the Southern California Computer Society will be
held this Sunday (January 18TH) at TRW. The exact location is
Building S (the cafeteria), which is located off of Compton near
Aviation in Redondo Beach. There should be a sign on Compton telling
you where to turn in and where to park.
Doors open at 10 am, refreshments are served at noon and speakers
begin at 1 pm. Speakers are Jerry Silver (Introduction to mini
Software) and Dr. John Titus (Microcomputers). Before adjournment at
4:30, there is some discussion from the floor, which last meeting held
some 600 people.
The reason doors open even before the Superbowl is over is to give
people a chance to talk with exhibitors. There is a $10 fee for
membership and probably some entrance fee for non-members.
If you have further questions, call Don Tarbell (SCCS president) at
Hughes (648-0116) this week.
-------
12-JAN-76 17:21:57-PST,463;000000000000
Mail from USC-ISIB rcvd at 12-JAN-76 1721-PST
Mail from USC-ISID rcvd at 12-JAN-76 1719-PST
Date: 12 JAN 1976 1719-PST
From: LEVIN at USC-ISID
Subject: What SCCS is.
To: bboard at ISIB
cc: carlisle, mann, moore, richardson
The Southern California Computer Society is a new group oriented toward
"home-grown" personal micro computers. The exhibitors will be
micro-computer manufacturers (MITS et al) and people
from The Computer Store.
-------
[Geoff]
-------
∂12-JAN-76 1956 LES Manna Proposal
I understand that you are generating a blurb in support of Zohar
having PI status with respect to his ONR proposal, for consumption
by H & S and Massey. Is this your impression too?
Name your fiction; I'll endorse it.
∂12-JAN-76 2316 LES Fiction?
Perhaps I am missing the context of your last message. What is it
that you will endorse?
∂13-JAN-76 0934 LES
If your earlier cryptic message was a request for prose regarding the
Manna proposal, I hereby offer the cover memo that I sent in with it
in the first place.
I think your note should be a bit more positive, saying perhaps that
you recommend he be "accepted as PI because of his unique and proven
capability in this area", or something equally flowery.
∂MEM Robert Floyd, CSD Chairman$$Manna Proposal to ONR$23 December 1975∞
Enclosed is a proposal to ONR listing Zohar Manna as Principal Investigator.
Since Zohar is not currently a member of the Academic Council (though he
once was and might be again) this requires a specific waiver of
the University PI policy, per Bill Massy's memo of March 7. I believe
that it meets the four criteria that Massy specifies. As I understand it,
the waiver needs to be approved by you, H&S, and Massy.
The proposal is mainly to cover a joint effort with Richard Waldinger of
SRI in writing a book, though for governmental bureaucratic reasons that
point is not made clear in the text.
As you may note, the budget calls for a reduced indirect cost rate on
the SRI subcontract. This was based on informal guidance from the
Sponsored Projects Office.
.skip 3
Enclosures: Proposal Routing Sheet, Manna proposal, SRI subcontract proposal
∂14-JAN-76 0125 LES Graphical Language paper
If you haven't read that thing I wrote, there is a substantially
different version in GRAPH.XGP[E,LES]. If you have read it, I
am interested in feedback.
Incidentally, after writing the last version, I just realized
that the "graphical machine" that executes the symbol strings
should be a Lisp machine. I haven't put that in yet.
∂14-JAN-76 0953 FTP:Waldinger at SRI-AI zohar as principal investigator
Date: 14 JAN 1976 0954-PST
From: Waldinger at SRI-AI
Subject: zohar as principal investigator
To: jmc at SAIL
cc: zm at SAIL, les at SAIL
dear john:
we need a memo from you recommending that official
university policy on principal investigators be waived in order that zohar
can be p.i. of our book contract. i understand that onr is virtually
ready to send us a check, but cannot until they receive a formal proposal.
here is a suggested text for the memo:
dear sirs:I recommend that official university policy against non-faculty members being
principal investigators be waived to allow Zohar Manna, Research Associate at
the Artificial Intelligence Laboratory, to be Principal Investigator of a
proposed project to be funded by the Office of Naval Research, involving the
preparation of a textbook. This step is justified on the following grounds:
1) Dr. Manna has been an Assistant Professor at Stanford before, and is thus
clearly qualified to be a principal investigator. He is also likely to be
promoted to a position where he will be eligible again.
2) He is the only one qualified to be the Principal Investigator of the
project: no faculty member could supervise the work.
Sincerely,
John McCarthy,
Director
Artificial Intelligence Laboratory
i spoke to les, who told me what the important points in the memo
should be. cheers, richard
-------
∂15-JAN-76 1357 REM DISK ALLOCATION, REFORMATTING, SPINDLE...
To: LES, JMC
One problem that has existed for many years is that whereas several of
the programmers agree that "word count" is not the right way to
allocate disk space and test for purging, the "bureaucracy" has
maintained the word-count policy. If a spindle program exists, people
will be punished for using it because of the word-count increase, unless
the policy is changed in the direction of actual disk blocks used.
As long as people are punished for using a program, they won't use it,
unless they are ecology freaks in a masochistic sense. Better to reward
people who spindle their files or submit to having it done for them.
Yes, we'll fix that.
∂15-JAN-76 1659 REM
no mandelbrojt yet, but a J.P.Mccarthy with cruncher much like your idea.
The difference between jpm's scheme and yours seems to be only that his
does look-ahead during crunching to find longer strings separated from
cursor by a few characters, thus if ab and bcde are both tokens then
the string abcdef will parse in his system as a-bcde-f but in yours
as ab-c-d-e-f. Yours is easier to program, and might give equally-good code.
∂15-JAN-76 2218 ZM REVAL - conclusion
I studied your REVAL program. It seems to be correct.
I have now no doubts that Pacini's algorithm is identical to
Vuillemin's delay rule. The trick is simply to do outermost substitutions
(call by name) and delay unnecessary inner subst. , rather than the
leftmost-innermost substitutions (call by value) of LISP.
Please leave me a copy of Wegbreit`s paper in my mailbox.
I am going to Atlanta ( NSF committee meeting and Principles of Programming
Conference) and to Austin [ the trip is paid by NSF ] , but I shall be
back on Jan 28 - before the ARPA visit. Zohar
Patte, pleae check into this:
∂16-JAN-76 0945 FTP:PRATT at MIT-AI
Date: 16 JAN 1976 1243-EST
From: PRATT at MIT-AI
To: JMC at SU-AI
My secretary defines 25FG characters as new math symbols not in the
SAIL set arise in my papers. Only takes a few minutes per character.
-------
∂16-JAN-76 0952 LES III Negotiation
I have arranged for John Poitras (and maybe Niels Reimers) of the
Stanford technology licensing group to come here to talk about the
III deal on Monday afternoon at 2 pm. If that is not convenient for
you, we can reschedule. Please let me know either way.
2π Monday is ok.
∂16-JAN-76 1108 RAK Refugee jobs
I spoke with Chuck Dickens at SCIP. He is still willing to consider helping,
and suggested as one possibility that instead of providing jobs SCIP could
give a "pre-job start" introduction to U.S. style commercial data processing.
He has not yet heard from the guy who he gave the problem to--the guy was
on vacation--so I said I would check back in about 2 weeks.
Ray Wilbur at HP is a personnel type. He seems to be little inclined for
public spirited reasons to get involved, but is willing to talk about the
general issues somewhat.
Sharon Balsgen was gone for this week. She will be back Mon (1/20).
I will be gone next week(1/20-1/24). Will follow up or talk to you on my return.
--Dick Karp
∂16-JAN-76 1620 FTP:CARL at MIT-AI
Patte: Here is a message from Carl Hewitt.
Date: 16 JAN 1976 1918-EST
From: CARL at MIT-AI
To: jmc at SU-AI, rwg at SU-AI
For mathematical papers the favorite Roman font for text is
Baskerville with mathematics in News Gothic Bold. However,
many prefer a gothic text font which is not avaiable here in
30 height. So you must choose between a 25 height or a 31 height.
Good luck,
Carl
-------
∂17-JAN-76 2244 REM via AMET PLANS (PRELIMINARY) FOR CRUNCH/SPINDLE
(1) We send in the various forms Friday, work should start officialy
on Monday if personnel dept accepts it. Meanwhile I am doing
preliminary organization.
(2) CRUNCH.PLN[1,REM] contains my current idea about how to do various
things. Anything in that file will be implemented if no rebuttal has
been fed back up to the time it is next to do.
I don't know any techniques, but I read that people devise algorithms
∂18-JAN-76 0121 100 : REM via AMET YOUR MESSAGE OF JAN 14
You mentionned that the problem of constructing concordances has
been studied, in case I have trouble with it. Could you give me
a brief idea what techniques you know about before I decide whether
to do it in a simple straightforward way or in a more efficient way?
for it. I'll enquire.
∂18-JAN-76 1301 100 : REM RE CONCORDANCES AND CORD-COUNTS
Then, I'll just proceed to devise my own algorithm unless I hear something
more definite from you in the way of an alternative. The first actual
programming will be to generate statistics of character-strings, to decide
which ones we will be using. (Scanning a file to produce the code, given that
we have already decided on the tokens, should be fast enough to be done
separately on each file, unless I discover otherwise in the future.
Meanwhile we must decide on the tokens to be firmwared in the programs.)
I am doubtful that scanning the file to produce the code will pay off,
so please check out fixed code also.
∂18-JAN-76 1556 REM via AMET PROCESS OF COUNTING TOKENS TO DECIDE ON CODE
(1) DECIDING WHAT TOKENS, I.E. HOW TO PARSE A FILE
**(2) COUNTING TOKENS IN LEFT-TO-RIGHT SCAN, GENERATING CODE, WRITING TREE
(3) CRUNCHING USING THE TREE TO DEFINE THE CODE
(4) UNCRUNCHING USING THE TREE TO DEFINE THE CODE
These are all separate operations with little interface, so I will do
the programming so that (2) can be tied to either (1) or (3) at a
later time. I suspect it should be done separately on each file,
i.e. tied to (3), but you may be right about tying it to (1) instead.
Actual experience with the particular code system to be used should
resolve this question after we get the first three parts working.
∂18-JAN-76 1854 SGK Usage
Since it seems the accounting system was at fault for my large bill
last month I have re enabled TVLOG. It will take 2.8 mins of time
per day (3.5 seconds each half hour) in its current state.
∂18-JAN-76 1952 ME Creating files while in E
You can switch to a non-existent file while in E and have the
file created by following the filename with /C (for Create).
For instance, αεFOO/C<cr>.
Also, MAILing from within E to a non-existent file should work.
The file will be created unless it is on a disk area protected
from you. Remember that to mail to a file, you must precede
the filename with a number sign (#), eg, MAIL #FOO
will mail to the file FOO.MSG.
Thanks.
∂19-JAN-76 1125 BG John McCarthy's seminar
To: PEOPLE.SEM[DOC,BG]:;
The seminar will meet at 5:00 p.m. on Monday, January 19. Chris Goad will
continue speaking on an axiomatization of knowledge, this time applying it
to the problem of the three wise men.
∂19-JAN-76 1138 REG
To: JMC, LES, TED
Dart crashed again because the tape drive didn't see the end of tape foil
marker. This is the fourth repetition of this incident since September 8.
No further DART dumps can be made until the various data structures
are corrected, which usually takes me about 90 minutes. While people
remain too busy to seriously consider replacement or proper repair of
the existing tape drives, I shall remain too busy to fix DART.
What about it, Ted?
∂19-JAN-76 2324 JJK
Dear Professor McCarthy:
Do you have a speaker and a topic for a talk at the computer
forum?
∂20-JAN-76 0655 REM SO-FAR -- CRUNCHING PROJECT
RU CRU1[1,REM] -- TYPE EITHER A NON-"T" CHARACTER OR ELSE A "T" FOLLOWED
BY THE LETTER YOU WANT TO BEGIN TRIGRAPHS.
DEFAULT FILE IS PCHECK.WD[UP,DOC] -- AT IFNAM IS A 4 WORD BLOCK YOU CAN
MODIFY IN RAID IF YOU WANT A DIFFERENT FILE.
THE PROGRAM AS IT NOW STANDS MIGHT BE USEFUL FOR GETTING AN IDEA WHAT
THE USUAL DIGRAPH/TRIGRAPH FREQUENCIES ARE AROUND HERE IN FILES.
I WILL BE IMPROVING THE PROGRAM TO GENERATE A COMPLETE LIST OF MULTIGRAPHS
OF HIGH-ENOUGH FREQUENCY. CRUNCH.PLN[1,REM] HAS OTHER STATUS OF PROJECT...
∂20-JAN-76 1307 RF
To: JMC, DBL, DRB, DES, RF, raphael @ SRI-AI
Reginal Bublil and Selma Light, from the Bay Area council on Soviet
Jewry, are going to be speaking at Stanford on their experiences
during their trip to Russia. They will speak at the Hillel-sponsored
bagels and lox brunch on Sunday, Feb. 1. The brunch itself starts
1200 (a good deal: $1.25 gets you a bagel with cream cheese and lox,
something to dring, and a sweet roll. If there are seconds, they are
either 25 or 50 cents.), and they will begin to talk about 12:30 or
so. It is at Bechtel International Center on campus.
02-01 noon, Bechtel Center, Bay Area Council on Soviet Jewry
∂20-JAN-76 2102 100 : Roode DECsystem-20
DEC is presenting their 20 to Gene Franklin and 1/2 of SCIP tomorrow
in the Durand Bldg. room 301 on campus. If you were to come, it might
be helpful,since you are a 10-proponent. It is at 1:30 in the
afternoon.
∂20-JAN-76 2203 DEW proposed chess project
There's a new version of my paper in your Wilkins file. You may (and should)
throw out the older one. Dave
∂21-JAN-76 0402 REM via AMET CRUNCH, PROGRESS REPORT
CRU1.DMP[1,REM] can now tally and report all digrams above a certain
threshhold (counts are sorted and the top 5/8 median are used to
determine where the threshhold will be), then extend all of those to
all trigrams above the same threshhold. I am now (detached job at this
very minute) processing UUO.ME[S,DOC]. I will also do MONCOM.BH[S,DOC]
and NOTICE.[UP,DOC]. What other files would you like studied
to get statistics?
Get statistics for HOTER.ESS[ESS,JMC] and MENTAL[F75,JMC].
∂21-JAN-76 0421 REM via AMET RESULTS ON UUO.ME
See the file CRU1A.LST[1,REM] -- There a many bigrams that exceed the
cutoff, but considerably fewer trigrams that do. It appears that
extending to 4-grams will not result in great savings. Thus a
cruncher that handles tokens of 1, 2, and 3 characters looks optimal
within our first general scheme.
∂21-JAN-76 0811 TAG
To: REM, RF, JMC
THE TTY SCANNER IS MAKING OUTPUT ERRORS. TED AND/OR I WILL
WORK ON THE PROBLEM TODAY.
∂21-JAN-76 1100 JMC*
CHOMSKY AND CAMPUS REPORT
∂21-JAN-76 1340 SGK
DS and I have come up with ideas in QUAD[105,SGK]. Your comments and suggestions
about the project, and the writing are sought.
My comments are p.3 of your file.
∂21-JAN-76 1536 100 : REM via AMET HOTER.ESS MENTAL
HOTER.ESS is a small file. If I lumped them together they
would make a medium (not large) file. Do you want them separately or together?
I don't care.
∂21-JAN-76 2015 RS via IMSSSS Your manuscript "Ascribing mental qualities to machines"
Yes, I would love to see it. Thanks for the offer.
Bob Smith
∂21-JAN-76 1106 JMC
Get statistics for HOTER.ESS[ESS,JMC] and MENTAL[F75,JMC].
[REM -- Done, see CRU1B.LST[1,REM] for TTY output for concatinated files]
∂22-JAN-76 0818 REM RESULTS (DIGRAMS AND TRIGRAMS) SO FAR
See CRU1*.LST[1,REM] where * is a letter A,B,C,D,...,H so far.
I have surveyed the ones I mentionned, the ones you wanted, and a few others.
Assuming that the first cruncher will use 1,2,3-letter tokens only,
we have the data and should now sit back and decide which tokens
to actually use among those outputted to the CRU1*.LST files.
After we have decided, then I write a program to scan various files
according to those tokens and come up with true counts of left-encoding tokens.
I'm I haven't yet heard from HVA whether personnel actually hired me.
∂22-JAN-76 0910 LES Russell-Carlstrom Visit Rescheduled
To: JMC, DCL, CCG, TW, DDG
Bill Carlstrom of ARPA just called to say that because of scheduling
conflicts on the 29th, he and Russell would like to come on Tuesday,
January 27, beginning about 9am for essentially all day.
Carlstrom's main interests are Formal Reasoning, Natural Language
Understanding, and Computer Vision, so he will want to spend more
time on those. Dave Russell will likely want to hear something about
Program Understanding and Program Verification, but probably not as
much. Russell may leave early.
The suggested order of discussion is as follows:
Formal Reasoning
Program Verification
Program Understanding
Natural Language Understanding
Vision
Carlstrom talked at length about his problems in selling our program
and the need, from his viewpoint, for our program to look more
symbiotic.
∂22-JAN-76 0937 REM
I have edited CRUNCH.PLN[1,REM] to be up to date.
∂22-JAN-76 1023 REM 3-LETTER STRINGS
There weren't very many 3-character strings that turned up in
CRU1*.LST, so I collated them by hand. The following were in all
5 files CRU1(A,B,C,D,G).LST -- " ", " an", " th", " to", "and", "he ",
"is ", "the". In 4 files: <tab>" ", " co", " in", " is", " of",
". ", "er ", "ing", "nd ", "of ", "to ".
In 3 files, not sure whether to include them: " a", "s ", "ed ", "ent",
"es ", "ion", "or ".
∂22-JAN-76 1435 BG
I added a declaration of ε and a more-general declaration to HEAVY.SET. Bill
∂22-JAN-76 1508 100 : queenie COMPUTER FORMUM
PROF. CERF CALLED TO ASK IF YOU COULD PROPOSE AN ALTERNATE TO GIVE A TALK AT
THE COMPUTER FORUM, AS DAVID LUCKHAM WOULD RATHER NOT HAVE TO GIVE TWO TALKS,
HE ASKED THAT YOU CALL HIM, X71365. THANK YOU.
Luckham didn't tell me he was already talking. How about Zohar Manna.
Well I guess I'll talk myself since you already have all the other
people. Title: Current issues in AI.
∂22-JAN-76 2253 100 : REM via AMET Reply to your question.
No, depending on how much overlap occurs the compression ratio could vary.
When we decide on which tokens to use during parsing, and actually perform
a parse accordingly, then we can compute a few c.r.'s and estimate the
c.r. for similar files. Since you are eager to get estimates, I will
use the data I've collected already, set up semi ad hoc a preliminary set
of tokens, and write the program that parses accordingly and tells us
what the correct statistics are for left-parsed non-overlapping tokens,
from which the c.r. can be computed easily. In a few days we should
have an estimate ready. -- Regarding types of text files, do we want
one system for all text files, or separate ones for (1) documentation
and essays (2) .FAI (3) .SAI with subdivision of (1) according to whether
they are source-manuscript (unjustified with POX/PUB commands) or ...(system too busy to complete message)
I think there should be several, but would like to see first the
one suited to documentation and essays in original manuscript.
∂22-JAN-76 2311 FTP:REM at MIT-ML
Date: 23 JAN 1976 0211-EST
From: REM at MIT-ML
To: JMC at SU-AI
...(CONTINUATION OF MESSAGE)... or fully-justified text including PUB-LPT output.
I will currently concentrate on documentation/writeup/essay files, but
not subdivided for the present. i.e. the first-estimates I will come up
with this weekend will be for the concatination of UUO.ME,MONCOM.BH,HOTER.ESS,
MENTAL,NOTICE,IL.TVR, and using the huffman code from that
multifile applied to other files in the same general category.
-------
∂22-JAN-76 2328 TAG
FINE. HOWEVER, SINCE THE CAMPUS COMPUTER CENTER INTENDS TO RETAIN
THEIR 7 TRACK TAPES, AND SINCE I HAVE TO USE THEM TO GENERATE A CARD DECK
ANYWAY, THERE IS NO ADVANTAGE TO USING SUMEX'S TU30S. IF YOU FIND A NEARBY
NET OUTFIT WITH A CARD PUNCH, HOWEVER, PLEASE MAKE A LOUD NOISE IN MY
DIRECTION.
∂22-JAN-76 2339 TAG
THAT FIGURES. DO WE HAVE A COURTESY ACCOUNT THERE? I HAVE TRIED, BEFORE,
TO CONTACT A FELLOW NAMED BOB LINEBARGER ABOUT A TIP PROBLEM, AND FOUND OUT THAT
HE IS CHIEF OF 360 OPERATIONS, OR SOMETHING LIKE THAT, AND THAT HE NEVER ANSWERS
HIS ADVERTISED PHONE OR HIS NET MAIL. THE AMES 67 WOULD BE NICE TO USE IF I
CAN GAIN PHYSICAL ENTRY TO PICK UP DECKS, AS IT IS BETWEEN HERE AND AMDAHL. IT
WOULD BE LOVELY TO DEVELOP A CATALOGUED PROCEDURE WHEREBY A PERSON HERE SAYS
"DOIT" IN SOME SIMPLE WAY, AND THEN JUST DRIVES BY AMES TO PICK UP A DECK TO
DROP OFF AT AMDAHL, INSTEAD OF HAULING A MAG TAPE OVER TO THE CAMPUS CENTER AND
WAITING AROUND.
∂23-JAN-76 0241 ME Imlac problem
Looks like the system is ok. It sent the Imlac the right stuff
but the Imlac didn't interpret it correctly. Perhaps your Imlac
needs to be reloaded.
That was it.
∂23-JAN-76 0338 FTP:REM at MIT-ML
Date: 23 JAN 1976 0639-EST
From: REM at MIT-ML
To: JMC at SU-AI
SU-AI is crashing so often tonight that I can't even send a message to
someone, so I'm going to bed instead of working. Maybe tomorrow morining
things will be working again... bye.
-------
The bugs are from a new system, so they may last a day or so.
∂23-JAN-76 0927 LES FEEPME
To: ME
CC: REG, JMC
To summarize our discussion, I would like to be able to request that
the system FEEP me any time a program either stops or enters TTY IOWQ
after computing for, say, 30 seconds or more (real time, not CPU time).
Because not everyone may want this feature, it probably should be under
control of an escape sequence and the option.txt file.
∂23-JAN-76 1026 LES ARPA games
To: TW
CC: JMC
I realize that you have less incentive now to spend time with them.
I imagine that you plan to quietly withdraw from their program and
utilize the other support that is available to you. This sounds
sensible to me too, but I would appreciate it if you would execute
this plan carefully. In particular, if you just plain say you are
quitting, it will probably cost us some money. If possible, keep up
a good front until the current contract ends (17 months from now).
∂23-JAN-76 1243 QIB RICHARD WEYHRACUH
HE CALLED AND SAID HE HAS A HORRIBLE SORE THROAT AND FEVER AND JUST
DIDN'T FEEL UP TO COMING IN TODAY. HE THOUGHT HE'D GO GET A THROAT CULTURE.
∂23-JAN-76 1635 REM via AMET Results of library search
To: JMC, RWG
Task, look for articles by Mandelbrojt on data compression.
PHASE 1 -- Comp.Sci. libr. a week ago.
There is no single index, a shame considering the field.
A partial search, looking for Mandelbrojt (or simila nam) and keyword
"data compression" resulted in nothing by Mandelbrojt at all,
but a rew misc. articles on Data Compression, including
one by a J. McCarthy of Switzerland quite similar to what our JMC wants.
PHASE 2 -- SCIENCE CITATION INDEX AT LANE MEDICAL LIBRARY, TODAY
Looking in source index for MANDELBR. (the permuterm subject index is
only at the physics libr) -- I discovered nothing on data compression,
but there was an article by Benoit Mandelbrot (no "j" in spelling)
of IBM Thomas J. Watson Research Center, "How long is the coast of
Britain -- Statistical self-similarity and fractional dimension"
In which the formulas D=-LOG(N)/LOG(R(N)) and L(G)=M * G ↑ (1-D)
originally invented by Le... F. Richardson, were used to show that
the coast of Britain is of dimension 1.25, of South Africa 1.05, and
the German Empire long ago some number inbetween I didn't write down.
(I looked up the 3 page article and read most of it) science 156 p636 (1967)
Well, that's the Mandelbrot I meant, but now that I think about it,
maybe his work in coding is obsoleted by Huffman.
∂23-JAN-76 1649 REM via AMET
It is possible you merely got your names mixed up between dimension and crunch.
I don't think so. I think Mandelbrot did some theory on encoding natural language.
∂24-JAN-76 1525 LES The next operating system
To: JMC, REG, JBR, ME
Now that DEC has announced TOPS-20 and outlined its properties, we
should begin reviewing our plans. The main alternates seem to be:
1) continue development of our existing monitor, making incremental
improvements;
2) plan a transition to TOPS-20, by inserting features that we need;
3) revitalize our work on the design and development of a new operating
system.
My current choice is #3, but we may have to settle for #1 or #2.
Overall, TOPS-20 seems to be a slight improvement on both TOPS-10 and
TENEX, though it seems to carry forward some mistakes (e.g. making
JSYS the dominant monitor call) and probably invents some new
mistakes that we don't know about yet. It lacks some important
features that are available in other systems. For example:
1) recursive file structure, so that directories may point to subdirectories,
etc.,
2) natural process linking at the command level, a la Unix,
3) display service such as ours.
TOPS-20 has a file protection scheme that is an elaboration of the one
we have, but is not as good as another scheme that is basically simpler.
Suppose that instead of having "built-in" protection data structures
and programs, that each file has associated with it the name of a
"protection file". This file contains a program that, when presented
a file access request, decides whether or not it will be honored.
It can look at the kind of request, the identity of the requestor,
the time of day, the phase of the moon, or whatever. Of course, the
protection file itself has an associated protection file, which may
be the same or different. I believe that this scheme is superior to all
existing protection schemes and is simpler to implement.
You probably have some pet ideas that you would like to try too.
The question is, can this all be wrapped in a package that someone
will buy? As usual, ARPA is a possibility. This has a somewhat
applied character, which seems to be one of the directions in which
they are trying to push us. Perhaps we should feel out Russell when
he visits on Tuesday.
∂24-JAN-76 1929 RWW ARITHMETIC
Declarations in ARITH.DEC[FOL,RWW]. For + association is to the right?
i.e. 3+3+3=3+(3+3) if you want it the other way just switch L and R
in the declarations.
rww
Thanks for arith.dec.
∂25-JAN-76 1707 REF via CMUA
I requested of the admissions secretary that my apllication
for admission to the Ph.D. program be reactivated. If it is appropriate,
I would apprieciate your recommendation letter. If there is anything
else i should be doing in this matter please let me know.
I hope things are going well in your life. They're keeping
me busy this semester (TAing, among other things). Let me know if there's
anything you'd like me to do.
bob
∂25-JAN-76 1745 LES Label Printer
We recently requested two Selectrics via government surplus that,
if we get them, will be free. We should probably await the outcome
of this request before pursuing alternatives.
Seem unlikely to me.
∂25-JAN-76 1749 LES
∂24-JAN-76 2149 KRD
MM (Minksy) is logged in via 493-5505; running BASIC. Sounds unlikely.
Is he in the neighborhood, or did someone perhaps snarf his password?
∂25-JAN-76 2026 LES Manna PI memo
There is a draft of a memo recommending PI status for Manna in
MANNA.PUB[E,LES]. I have also placed a printed version and the
memo from Floyd on your terminal.
As I mentioned earlier, it is apparently important that we get
this going. ONR has the money and is willing to fund it, but
unless they can commit it very soon, it may go away.
∂26-JAN-76 1139 JH SENSE
Hi John,
The poster for the last sense meeting is on SENSE.PST[2,JH].
I would be willing to do the publicity if you can't get anyone else.
I am not at the lab much these days, so that makes it somewhat difficult
for me to handle publicity. My phone numbers in Berkeley are at UC
642-7453 and at home 549-1411.
Hieronymus 642-7453 and at home 549-1411.
Jim
∂26-JAN-76 1908 FTP:CERF at USC-ISI (Response to message)
Date: 26 JAN 1976 1730-PST
From: CERF at USC-ISI
Subject: (Response to message)
To: JMC at SU-AI
In response to your message sent 22 JAN 1976 1913-PST
I will advise Sally Burns. I don't recall whether yur name appears on
the current agenda, but there is no problem announcing a change.
Thanks.
Vint
-------
∂27-JAN-76 0701 REM FIRST ESTIMATE OF CRUNCHING COMPRESSION RATIO
UUO.ME[S,DOC] AS INPUT FILE FOR TALLYING UP THE LEFT-PARSE,
HUFFMAN TREE DONE ON RESULTANT TOKEN COUNTS,
INPUT BITS = 4544204
OUTPUT BITS = 2754397
OUT/IN = C.R. = .606 (I.E. 61 PERCENT, I.E. 39 PERCENT REDUCTION)
THE TOKENS WERE ORIGINALLY SELECTED BY HAND FROM LISTS OF
COMMON TOKENS IN VARIOUS FILES.
(THE ABOVE TALLY ASSUMES THE PROGRAM-ADDITION I COMPLETED THIS
MORNING IS WORKING OK. SINCE IT PRODUCED REASONABLE RESULTS,
ONE CAN BELIEVE IT IS CORRECT FOR THE MOMENT)
∂27-JAN-76 0735 REM MORE FIRST-RESULTS
HOTER.ESS[ESS,JMC]
BITS IN = 166453
BITS OUT = 97354
C.R. = .585 (I.E. 58 1/2 PERCENT -- I.E. 41 1/2 PERCENT REDUCTION)
I am disappointed that there wasn't more compression. Can you
tell me a file that tells what tokens were used and what codes
they received so that I can readjust my intuitions or make
suggestions.
∂27-JAN-76 1508 100 : REM via AMET Reply -- Files containing info on parse...
The files UUO.LST,HOTER.LST[1,REM] contain the actual parse counts
that occur when the files are parsed using TOKENS.DAT[1,REM].
There is no file containing the code lengths, what you wanted, but that
can be obtained incrementally by poking around with RAID...(system too busy to complete message)
∂27-JAN-76 1546 FTP:REM at MIT-ML
Date: 27 JAN 1976 1836-EST
From: REM at MIT-ML
To: JMC at SU-AI
By poking around in RAID after the program has eaten PARSE.DAT and
used it to compile the Huffman code in-core, after it has typed out
the total number of bits in and out and gone into RAID itself. If
you want, I can have it dump out the Huffman code in readable format
before goint into RAID, but for the moment here is the internal
representation: goint into RAID, but for the
oment here is the
internal representation: FIRHUF: points to first word of first cell
of terminal nodes. SECHUF: points to first word of first cell of
juncture nodes. TOPHUF: points to first word of last cell of
juncture nodes, the trunk node. All cells are three words in length,
thus [SECHUF]-[FIRHUF] is multiple of 3. Terminal nodes are in the
following format:
word 0 -- ASCII string of input token
word 1 -- left half number of bits in ASCII string, one of these: '07 '16 or '25
right half number of bits output in crunched file for that token
word 2 -- 400000,,0 bit internal flag you may ignore
377...,, bits (as many as specified earlier) actual output bits
0..,,... bits (right-justified) count of this token
(Because entropy approximately equals number of output bits, those
two parts don't bump into each other even though worst case for
each of them is larger than 18 bits in a large file -- it is a
general property of such codes that the number of bits needed to
show output code plus the number of bits needed to show count
of occurrance is a constant equal to logbase2(size of file) appx.)
If you want to examine a nonterminal node (juncture node), the first word
is a pair of halfword pointers to the two son nodes, the first half of the
second word is zero, and the rest is the same as above.
It would be interesting to try my History-tree crunching method, using a
similar number of tokens effectively, on the same files, to see how the
two methods compare. At your service...
-------
Yes, try History-tree crunching.
∂27-JAN-76 1917 REM Reply about "qu" and "the " in HOTER.ESS
"qu" is not considered because it does not occur the required 280 times
in combined HOTER.ESS and MENTAL
"the " is not considered because it is longer than three characters,
the program is at present only capable of extending bigrams to trigrams.
ADDENDA -- "qu" occurs only 104 times, hence there are an awful lot
of other bigrams that ought to be included before "qu" is considered.
∂28-JAN-76 0148 FTP:Omalley at SRI-AI Next BAAIC meeting on 2/18
Date: 28 JAN 1976 0146-PST
From: Omalley at SRI-AI
Subject: Next BAAIC meeting on 2/18
To: BAAIC-People:
We missed a few there but for February, Marvin Minsky is going
to talk at Berkeley on "The Problem of Coherence in Artificial
Intelligence."
Dinner will be at 6:00 (1800 for those supported by ARPA) at King Tsin,
1699 Solano Ave., Berkeley. The cost will be $7.00 unless too many
people object.
Oh yes, the date is 2/18 (wed.). There will be further notice but
this will at least allow you to put it on your calendar.
See you then.
Mike O'Malley RSVP
02-18 BAAIC 6pm King Tsin 1669 Solano Bkly, Minsky AI Coherence Problem
I expect be attend BAAIC in Feb.
-------
There will be two of McCarthy at King Tsin and at Minsky talk.
∂28-JAN-76 0326 REM THE DETAILS OF THE HUFFMAN CODE YOU REQUESTED.
See the file JMC2XX.LST[1,REM] for a complete listing of the code.
It shows input ascii text, number of bits input (7 14 or 21),
frequence count, number of bits output, actual bit stream output.
∂28-JAN-76 0412 ME
I squeezed out 340 (octal) words from E's lower segment.
∂28-JAN-76 0528 REG
To: LES
CC: JMC
∂28-JAN-76 0105 LES System Unusable
The system has been unusably slow the last couple of hours. I am suspicious
that something is sapping its efficiency.
[REPLY BY REG:]
In my opinion, the management has failed to provide the right
proportions of capital and labor. Moreover, the management has
failed to distribute labor in optimal proportions among the various
areas of endeavor. While it is people, not computers, that produce
the results we're after, if you apportion too little of the computer
to each person, as you have done, then all results are delayed.
Comparing the present situation to the one that existed in 1972 when
New Ampex came online, the capital situation has effectively improved
in only one minor way: the operating system is apparently 32K smaller
(further improvements arising from better utilization the additional
core memory will occur shortly). There have been other capital
improvements, like the 11/45 system, but nothing relevant to system
performance. Aside from the one improvement noted, the user
community has increased by some large factor while the hardware
(Librascope) has decreiptitated to a useless condition, and while the
systems programming staff has shrunk to nearly vanishing. In
addition a variety of computationally expensive processes have been
added to the system (PUB and the XGP) and you (i.e., the management)
have encouraged people to use, and possibly over-use them. Finally,
although we attempted to run a maximum of 48 jobs when swapping on
the Librascope, we are now allowing a maximum of 63 (and at the time
you sent the message above there were apparently 51 jobs).
I have been working on better utilizing the machine resources that we
do have available. However, because I'm short-handed I can't do as
much as I'd like to in this area; I am concentrating on those
projects that are going to pay off when the KL10 and mappiplexor
arrive.
If you're trying to start a flap over system performance, I'm not
having any part of it. If you want to make a contribution towards
amelioration of this problem, you could start by seriously
considering how to procure a swapper that works, and the additional
disk channel and controller that we'll need with the KL10.
∂28-JAN-76 0547 REM More of same for comparison
JMC3XX.LST[1,REM] has complete parse and Huffman code results
for MONCOM.BH[S,DOC].
Input is 2683149 bits
Output is 1640037 bits
C.R. = .611 (I.E. 61 PERCENT, I.E. REDUCTIN OF 39 PERCENT)
∂27-JAN-76 1305 JMC
I am disappointed that there wasn't more compression. Can you
tell me a file that tells what tokens were used and what codes
they received so that I can readjust my intuitions or make
suggestions.
[REM -- DONE, SEE NOTES EARLIER THIS EVENING]
∂28-JAN-76 0554 REM ONE MORE THING TO TRY
We might try adding more tokens to see if the code gets better.
This can be done by liberalizing the 5/8 median cutoff to 3/4 or more,
re-doing the generation of c{ntext-free t{ken parsing, c{llatin$
the t{kens t{ see which ones are very comm{n, then rerunnin$ the part
that does left-parse and Huffman treec{de t{ see if anythin$ is
different. There is a possiblility, hoever that it ill $et {rse, {t better.
[EXCUSE THE GARBAGING -- TTY11 IS STILL BROKEN]
∂28-JAN-76 2000 REM via AMET History-tree Crunching
The overlapping tokens with left-parse longest-string (CRU1.FAI) has the
disadvantage that the code is redundant, i.e. we are throwing away things
by having duplicate ways to encode (and selecting one of them).
I will try adjusting parameters to see if it can be made
better than 60% c.r., however.
The History method uses longest-matching left-context to decide the
current "mode" or "machine-state", but then uses a crunching scheme
at that point that doesn't have any overlapping codes. In the simplest
model, it crunches single characters only, but because of left-context
info it can usually guess almost the exact character to occur,
resulting in one bit output per character if things are super-winning.
More advanced schemes would sometimes crunch more than one character at
a time if the entropy for the first character is significantly less
than one bit, therby doing better than one bit per character in those rare
circumstances. Usually however there will be perhaps 4 possible characters
normaly occurreing in the particular left-context, resulting in 2 bits
per character. Of couse, more entropic data will do even worse, maybe
even as bad as he c.r.=.6 we got with CRU1.FAI -- we'll know
when I get a program running in a few days to compute History code.
One major modification to all these methods is to separate words from
delimiters. For example the phrase "for all x such that" might occur
frequently in a paper, but would be in various forms:
for all x such that
for all x such that
for all x such that
for all x such that
for all x such that
because of justifying and randomness in source document typing. Each
would be parsed as a completely different bunch of short strings now,
but separating out delimiters would allow the entire phrase to be
crunched as a unit. The delimiters would be even more crunchable,
with words removed, because long strings of these:
A A A A A
A A A A A
A A A A A
A A A A A
etc. would appear in widely separate parts of the document, often enough
to be common strings and be crunched as units. ("A" denotes a word, i.e.
a break to the words co-routine data-pipeline). By crunching the words
and delimiters separately, then collating them when uncrunching, I believe
any of our various schemes can be improved, but again nothing definite
until we try a scheme both ways. With POX or PUB source, it may even be
possible to split the data into 3 streams and crunch each separately,
text-words, text-delimiters, commands.
OK. All these approaches seem plausible.
∂29-JAN-76 0100 ME
∂28-JAN-76 2326 JMC
Am I mistaken, or is type out from E using "x ty" much faster
than type-out from NS?
[ME - I can't believe there is any difference. NS sometimes has to
read in another story before it can type out anything, and that may
cause you to have to wait a little before typing starts, but once it
starts, NS does the whole thing in one UUO, whereas E does several
UUOs. When you give an XTYPE command to E, however, it already has
the text in core, so it can respond quickly. Perhaps E is so slow,
that it never (or seldom) fills up the output buffer, so it doesn't
have to wait for output. NS might get swapped out while waiting for
the output buffer to empty. Also NS is 34 pages whereas E is (at
least) 8 pages (not counting the upper segment, which is now
locked in HICORE like other upper segments), so NS gets poorer service.]
∂29-JAN-76 0336 RHT
where is the letter?
∂29-JAN-76 1208 LES IMLAC & air conditioning
To: TED
CC: JMC, REG, TAG, HVA
The Dendral IMLAC is now ours. It is located in a temporary building
back of Serra House and suffers from a flakey memory. To pick it up,
contact one of the following at ext. 74878: Bill White, Bob Engelmore,
or Ray Carhart.
As you know, Plant Services is concerned about the additional heat load
of the KL10 and are asserting that we need more air conditioning
capacity. Of course, they were not competent to engineer the initial
installation and I have seen nothing recently that gives me greater
faith in their judgement. I propose that we undertake a small study
and perhaps seek outside consultation.
I suggest that you or someone add up the expected heat dissipation
of the system and compare it with the sum of the green monsters'
capacity and the "normal" heat load of the computer room. To
estimate the latter, consider the fact that the computer room was
originally intended to be an auditorium. Estimate the seating
capacity and multipy by 100 watts/person.
If we actually do need more capacity, I think that we can argue
that it is available from the central air conditioning system, since
it is not being called upon to cool the entire building, as originally
designed (i.e. the full circle). In any case, we need to know what
the expected load is.
I suggest that we simply stall until the KL-10 is installed and
see how hot the room gets and extrapolate to summer conditions.
∂29-JAN-76 1233 LES Kant phone
In preparation for the Datamedia terminal installations, we have looked
into home phone costs. Elaine Kant is in a remote location where it
will cost $25/month instead of the usual $8. Cordell says he thinks it
is worth it. Do you care?
No, but let's make Cordell feel a bit guilty about feathering the nests
of his graduate students. I guess I didn't understand that they
were all to get home terminals.
∂29-JAN-76 1426 RWW
ARE ANY STUDENTS USING FOL?
Yes
∂29-JAN-76 1500 LES Oppen reappointment
Derek Oppen is pressing for some feedback on possible continuation.
Oh well, let's do it.
∂29-JAN-76 1636 LES Subroutine library
The note that I wrote in trying to get the subroutine library set up
is in SUBR.LIB[E,LES]. I would want to change some things in it if
I were reissuing it, but you can get the idea.
∂29-JAN-76 2248 LES Edit & mail to processes
I like the Edit & Mail model of process initiation that we discussed
this afternoon, but your proposal does not go far enough. We got off
on subroutine libraries, which I agree are important but they do not
handle the I/O problem that I was talking about.
In general, one wants to be able to initiate a number of processes.
Some will go off and do their thing and require no further interaction.
Others, when they get in trouble, should return an error message and
die. Still others will require multiple interactions.
A good way to think about this within the editor model is that the
interaction with each process occurs on a different page. You can
display parts of various pages on portions of whatever screens are
available and can switch your keyboard to whichever page you want to
work on.
I advocate writing most translators as if they had only disk file io.
The command that initiates processes would be permitted to name a list
of processes. Arguments of the command constitute the input to the
first process, whose output feeds into the second process, etc.
[essentially the same as UNIX].
The thing that this buys you is io flexibility without having to
build all the io options into each process. For example, if you initiate
a file search that emits a list of names, you might like to have it
stream by on your display, or you might like it formatted in a five
column list, with numbered pages for output on the LPT or XGP or you
might want it formatted in a special way of your own choosing. If the
output process is separate from the list emitter, you can select or
write your own output process.
Mail could be the mechanism of interprocess communication. Then the
process string could be set up and continued by each process mailing
packets to its successor, which could be anywhere on the network.
∂30-JAN-76 0256 REM CORRECTION, MINOR
After writing most of the code for the History-tree method, and getting it
(so far) to work, I went back to see if the old parts of the program still
worked, and found a discrepancy, which I finally resolved:
The file JMC1XX.LST[1,REM] has an improper header. That file is for
HOTER.ESS only, not for that and MENTAL. The totals for HOTER alone
are IN=166453 $UT=97354 CR=.585 For HOTER + MENTAL, they are
IN=551558 $UT=334549 CR=.6065 -- Not a signivificancedifference
(PLEASE EXCUSE THE GARBAGING, TTY11 IS ACTING UP AS USUAL)
Some data on the History method will be forthcoming...
∂30-JAN-76 1948 100 : Sarah Overnight whereabouts
Sarah staying at a friends overnight and the number is
321-6416.
∂30-JAN-76 2035 REM Number base-conversion, emitting "less than one bit"
Did anyone ever tell you of my invention several years ago for
converting from any base into any other base? (RWG thought it was
nice when he heard about it) Basically a number representation is
a sequence of digits such that each digit subdivides the possibllities
and selects one of the subdivisions, which is then further subdivided by
the next digit, etc. creating a sequence of nested intervals whose
intersection is the REAL number being represented. In base n
the digits are 0,1,2,...,n-1 and the sequence . d1 d2 d3 ... can represent
any real number between 0 and 1. If the interval before a digit is
[A,B] and the digit is k then the interval after that digit is
[A+(B-A)k/n, A+(B-A)(k+1)/n]. Generalize to an arbitrary rule
such that before a digit is seen the division points between the
subintevals is known, then when the digit is seen which interval it is
is known. (Obviously this is the case in any one-pass coding system,
such as my file-crunch systems) The algorithm for converting from one such
system to another is to maintain upper and lower bounds for the real number,
usually these bounds will be rational and can be maintained as [N1,N2]/D
integers. Then whenever the known interval is a subset of some
output-system interval, that digit is emitted and the known-interval is
normalized as a subinterval of the output-digit interval. Numerically
it is a trivial algorithm. The only design problem is that the integers used
to represent the interval tend to grow as the process runs (except in
ses), especially if the input data produces a real-number...
∂30-JAN-76 2051 REM Continuation...
real-number that is almost exactly on a decision point for the
output system. There are a number of ways to handle this problem using
overlapping intervals (i.e. a redundant number system).
The application to file-crunching is that in the history scheme it is
often possible in certain left-contexts to almost exactly guess a character,
say with propability greater than 1/2, thereby requiring that less than
one bit be output to avoid wasting information density in the output file.
The usual situation is to crunch more than one character at a time, as
with your left-to-right parse scheme. An alternative is to actually emit
less than one bit, i.e. emit a subdivision that contains less than
one bit of information into a post-processor that converts this "funny base"
into standard binary, or overlapping slightly-redundant binary.
This is a research problem, not an easy algorithm, because (1) we don't know
what the propablities are until we have decoded the preceeding token
(2) numbers get large and must be pruned in a way that doesn't
lose information. But I think the method has merit. Probably the "best"
ascii-crunching scheme will be (a) history context for crunching the
next character (b) funny base conversion to allow less than
one bit to be emitted (c) regular re-sync to flush large numbers (d)
delimiter and word separation into two pipelines as I described the other day.
Meanwhile I proceed to work on the easier things for now...
∂30-JAN-76 2232 JJK Equipment for Computer Forum
To: DBL, BPM, JMC, levy @ PARC-MAXC, buchanan @ SUMEX-AIM
To: feigenbaum @ SUMEX-AIM
Please let me know if you'll need any sort of special equipment
for your talk, such as a projector.
I assume there will be a transparency projector, though I am not
sure I will use it. By the way, precisely when and where am I
scheduled to talk.
∂31-JAN-76 0617 REM HISTORY (LEFT-CONTEXT) SINGLE-CHARACTER HUFFMAN-CODE
To: JMC, REM
HOTER.ESS[ESS,JMC] C.R. APPX 44%
DETAILS OF THE CODE ETC. TO BE FOUND IN A LARGE FILE OF TTY OUTPUT, HH1.LST[1,REM]
In review, there is a list of special left-c{nteexes and f{r each character
in the file the actual left-context is compared against these, and the longest
matching left-context determines which code table to use for that one
character. The code table is a single-character huffman-c{de.
Regarding the first complete results for this scheme in HH1.LST, one really
neat thing was that the null hist{ry (i.e. crunchin$ the character that
follows any rare character) had entr{py {f less than 60%, and alm{ve a$$$$$$$$
almove all actual nun$$$$ n{n-null histories had smaller entropy, often
less than 30%, frequently less than 40%, almost always less than 50%.
[EXCUSE THE GARBAGING -- TTY11 IS ACTING UP LIKE USUAL]
A crunch ratio of 45% (As I understand it, this means that
the crunched file has 45% as many bits as the source.) is good enough
for practical purposes provided the overhead from the tables used
for the crunching doesn't bring the ratio above 50% or if a fixed
table can be used for all text files. One might consider a special
monument to justification in justified files where one doesn't distinguish
the number of spaces except after periods for lines that have 65 or
whatever total characters. I don't know if this would make a useful
difference. What does the computation time look like and the size
of the job that does the crunching and uncrunching.
∂31-JAN-76 1646 REM via AMET
Correction, add 1 to the % figure for HH1.LST and HH2.LST (bug in program).
∂01-FEB-76 1721 LES Jeff Rosen Account
Rosen's remark to you is accurate. I have no reason to believe that
his term project will impose any immense load.
You seem to be mushing into an unstated policy. If you wish to state it,
I will follow it.
∂01-FEB-76 1749 FTP:HQM at MIT-AI
Date: 1 FEB 1976 2047-EST
From: HQM at MIT-AI
To: jmc at SU-AI
I will be coming out to the stanford area around
the 14th. i was wondering if you could provide
a temporary user name for me
on your pdp-10
while i'm around. i don't need a disk directory, but
maybe some way of using your system.
I'm anxious to learn a time-sharing system other than
ITS, and also to learn something about the PC program
( i'm already familiar with the Drawing Prog., and
we recently got a copy of PC to run in our
DEc simulator...) . If its not too much strain on your
resources...
tnx.
Henry Minsky
-------
Yes, we can do that, and HQM will be fine. For how long do you want it?
The name will be for your personal use only - no friends, and we may have
to limit the non-trivial use of the PC program to slack hours. Depending
on when you come and how fast things go, there may be substantial downtime
due to installation of KL-10 processor.
∂01-FEB-76 2033 FTP:HQM at MIT-AI
Date: 1 FEB 1976 2331-EST
From: HQM at MIT-AI
To: jmc at SU-AI
tnx. i will be around about a week and a half.
Don't worry about me using system at prime-time,
I will limit myself to the
evening and wee hours of the morning...
-------
∂01-FEB-76 2035 BPM <MCCARTHY>@USC-ISI
The two files you had there are now ARCHIV.DIR and MESSAG.TXT, both on
[ISI,JMC]. They don't appear to be particularly interesting. The first is
an archive directory of all the files you had at ISI and the second contains
the message that you sent to yourself down there.
∂02-FEB-76 0349 REM via AMET C.R. compared with results of J.P.McCarthy
You are correct in your understanding of my c.r. figure. nbout/nbin
which for our 7-bit ascii is nbout/7. There is at present in my
.msg file (by the time you see this note, it will probably be moved
to CRUNCH.I[1,REM] where you may peruse it) a summary of data I got
from the article by J.P.McCarthy. A comparison between my 45%-50% c.r.
and the nbout/7 column (I converted from the 8/nbout figures he
was using, including also a column of nbout to compare with
Shannons estimates which he quoted), shows that my results are about
the same as his. Considering that we at SU-AI tend to throw in strange
characters, and unusual wordings involving strange sequences of ordinary
characters, whereas his "English Text" file was probably some plain
literature or news item, I agree with your estimate that my
scheme is acceptable providing that it runs fast. Since the tight loop
in the crunch loop can be done by character-dispatch table-lookup,
and since the logic for the uncrunch is single bit branching which can
be done on this computer by a TDNN or TDNE followed by a JRST etc., if
we use the same code for a class of files and pre-compile the
tables, we can just dump them in from DSK and both the setup and crunching
can be fast. But, of course, we'll see exactly how fast...
∂02-FEB-76 2238 FTP:KING at PARC-MAXC Computer Forum
Date: 2 FEB 1976 2236-PST
From: KING at PARC-MAXC
Subject: Computer Forum
To: jmc at SU-AI
In answer to your question, our technical session begins at 3:15
P.M. on Thursday, Feb. 5. Place: Durand 450.
-------
∂03-FEB-76 0544 REM MORE RESULTS, HISTORY-HUFFMAN, AND LIBRARY
The latest version of the program (this morning) has 3 new features:
-- Nodes with count less than 7 are merged together into an escape node
in which the actual character is quoted after the node itself is generated.
-- In the listings the program tells the amount of overhead needed to
specify the Huffman-Code at each history-node, and this overhead
is added in when computing the C.R. -- for grand totals, both the
partial C.R. (not counting overhead) and complete C.R. (incl. overhead)
are given.
-- Some extraneous output is now skipped over, and if you hit ↑O during the
dump of the raw counts (before Huffman code has been generated) the program
will re-establish output for the important part (the Huffman code and totals).
A full run of this program, using the standard file of left-contexts
to consider (not optimized at all for history-tree method, consists of all
the nodes used for your method, plus all high-frequency mono-grams), is to
be found in HH2A.LST[1,REM] -- input file MENTAL[F75,JMC] --
partial c.r. 49%, full c.r. 52% -- when I make listings of the output
and study them carefully, I should be able to improve the selection of the
left-contexts used, combining several if they have virtually identical codes,
especially after a space the preceeding character is irrelevant because of
varying number of spaces between words, and after a first-character of a word
the number of preceeding spaces is irrelevant, and splitting nodes that have
large count but do not appear to be of that type.
In the comp-sci library today (monday) I was reading random articles
from Information Storage + Retrieval that looked at all relevant, and I found
a reference to another article by Mandelbrot in AMS Symp. app'l Math., so
I looked it up and found some discouraging information in terms of using
a dictionary of words, in particular that in any large body of text,
half of the words only occur once each (half of the different words, that is)
meaning that some other method of encoding must be used most of the time,
such as the ones I have developed and the word-fragment method I read about.
See ISR vol.9 no.12 (1973) "Selection of Equifrequent Word Fragments ..."
by Schuegraf and Heaps, for the word-fragment method (as applied to ISR
mostly, rather than crunching per se).
Also I found an article by Marrion + deMaine in COMM ACM
that tells about their wonderful crunching scheme that achieves a c.r. of
61% on an English writeup file, thereby making my methods even more attractive.
But again we must use both methods on exactly the same file(s) to be sure
the difference is in the method rather than in the entropy of the data.
∂03-FEB-76 1026 HVA Computer Forum Events
To: JMC, LES, ALS, TOB
Sally Burns (7-1458), called re program for Computer Forum. She
needs to know today if you will be attending any or all of the
following:LUNCHES -Tuesday and Thursday at 12:15, at Faculty Club;
DINNER:Thursday (Feb.5),6:00 p.m.,Cypress Room,New Holiday Inn.
Would you please call her as soon as possible. Thanks.
∂03-FEB-76 1039 HVA CORRECTION -LUNCHES - Computer Forum
To: JMC, LES, ALS, TOB
The lunches are THURSDAY and FRIDAY (not Tues. & Thurs.) SORRY!
∂03-FEB-76 1149 ZM
To: JMC, RWW
Prof. Tom Maibaum (Un. of Waterloo) called me. He is going to be in
Palo Alto on April and offered to give a talk on
"Non-deterministic functions of higher types"
Should we invite him to give such a talk in the A.I. Lab. ?
Let's do it
∂03-FEB-76 1919 LES BBN Pager is Safe (I think)
To: JMC, TED
Got in touch with Steve Walker of ARPA and talked him out of snarfing
our Pager, I think. It didn't take much, since other parts of his plot
were already falling apart.
∂03-FEB-76 2012 REM via AMET "WORST-POSSIBLE" RESULTS ARE NOT TOO BAD.
NOTICE[UP,DOC] seems to be a "worst-possible" file for crunching
because of the large number of users who have written on all various
subjects over a period of several years and had their short files concatinated
with almost no effort to standardize the format or wording used. I thus
consider it good news that my run of the not-yet-optimized history method
on that file resultd in c.r. of only 53% with all overhead included (51%
not counting overhead). If we're favored by Murphy and Finangle, we may
be able to develop a single code that achieves 55% or less on all ordinary
text files (not incl. XGP files with their rubout sequences, only files
formatted for editing and/or line-printing) consisting of essay, writeup,
etc.
CRUNCH.PLN[1,REM] has been brought up-to-date again, summarizing
my results and plans in this project.
∂05-FEB-76 0529 REM Listing files of results so far
I plan to delete some of the *.LST[1,REM] files that are collected
TTY output from runs of CRU1 on various files, to conserve disk space.
I'll only purge those that are not interesting or which have saved
on dart dumps (T-style mostly). If any you want any of kept for you
to peruse on dsk, let me know...
Perhaps you should crunch them, but I suppose there isn't much redundancy.
∂05-FEB-76 1652 REM via AMET Those listing files
In the implicit heiarchial memory we have from DSK: to DART, the purpose
of crunching is to provide an intermediate point for those files that
are too important to allow 15 minutes wait restoring from DART but
are not important enough to keep in ASCII on DSK: all the time. The
listing filesthat I have already listed on LPT: are no longer worth keeping
on the DSK: at all unless you want to peruse them for some reason (once you
asked for detils of the code so you could better understand what was going
on, I don't know if you are still trying to increase you understanding, or
have reached your desired level). -- I might try crunching them (more
correctly, simulating crunching, i.e. just computing c.r. and code used if
they were to be crunched) except that their statistics are quite different
from ordinary text so that your scheme would fail horribly and my history
method would do moderately poorly.
∂05-FEB-76 1751 REM via AMET USING HH3A.LST (I THINK) AS INPUT TO CRUNCHER
I had to disable the CR-LF feature to allow bare linefeeds in the file
(the RAID session at the beginning has bare linefeeds) but then it
went through ok and got a c.r. of 44% using my history method.
∂05-FEB-76 1908 REM via AMET ADDENDA TO PREVIOUS NOTE
With program correctly modified to not cheat on bare linefeeds, i.e.
a crlf is always considered as two characters rather than one, so that
bare cr's and bare lf's can be legally considered, and with the SNEKOT.DAT
file updated to include digits, the c.r. comes out to be 39% with HH3A.LST
as input file.
Well, that's surprisingly good!
∂06-FEB-76 0002 REM NEW RESULTS, <=50% C.R. OVERALL
I spent this afternoon (evening?) going over the two listings
HH2A.LST (input file MENTAL[F75,JMC]) and HH3A.LST (input file NOTICE[UP,DOC])
weeding out those left-contexts which do not improve the code much beyond
what their parent-node does, and adding a few other nodes. On general
principles, I left in <tab> and all single-character letters and numerals,
even if they didn't have a code much better than the null-history (catchall
for left context not matching any of the other left contexes) code. The
result is SNEKOT.OPT[1,REM].
I then ran that set of left-contexes on the conglomerate of four
files: MONCOM.BH[S,DOC], NOTICE.[UP,DOC], MENTAL.[F75,JMC], HOTER.ESS[ESS,JMC].
I left out UUO.ME because together with MONCOM.BH it would tend to bias
the data toward official manuals, whereas most text files that are not
source files are probably more random like NOTICE.[UP,DOC]. The results are
favorable: input 4,893,xxx bits, output 2,373,xxx bits, overhead 24,xxx bits,
partial c.r. <49%, complete c.r. <50%.
If you agree that this history method is acceptable for implementation,
I can scan a larger collection of files in this mode and set up a standard
code to be used on such files (any writeup except XGP-formatted). Because
there are files fully-justified, not-justified at all, and all shades in
between, it seems unlikely that distinguishing them will be really profitable.
The problem of all-upper-case-files is partly solved with my method
because the one-letter left-context that is always present except at the
beginning of a word will select a different code appropriately, even if the
upper-case/lower-case ratio in the crunched file differs drastically from
that of the files surveyed. Source-files, however (.FAI .SAI .LSP .F4 ...)
won't be crunched well enough by a code designed for English writeups.
They will need their own code, possibly their own set of left-contexts.
The full results of the four files described above are in HH5I.LST,
and the pseudorandom text I generated immediately afterward is to be found
in HH5I.GIB.
I would like you to try one of the books WUTHER or GRIMM on LIB,DOC both
with your present set of nodes and one optimized for the book. This is
more to get an estimate of the condensation to be expected with books
than for the purpose of crunching these two. The amount of crunching you
get is acceptable, but how much computer time does the process take? I
talked to Huffman this morning, and he said he didn't know the current
state of the art but thought that 50% was about right for a practical
technique and referred me to Peter Neumann at SRI. Neumann thought that
perhaps we might be able to do somewhat better but wasn't sure and said
he would call me back if he had any suggestions.
You might also think about what the commands for crunching and
uncrunching and spindling should look like to the user.
∂06-FEB-76 1213 FTP:REM at MIT-ML
Date: 6 FEB 1976 1514-EST
From: REM at MIT-ML
To: JMC at SU-AI
The system went down so I came here to reply. I'm one step ahead of you
on GRIMM[LIB,DOC], I ran it about 6am. If I remember correctly, the
c.r. was 44% and the listing file is HH6A.LST[1,REM]. -- Yes, I guess
I'll have to propose a command syntax sometime soon, although it'll be
essentially what you suggested last month.
-------
∂06-FEB-76 1551 PLW ai lab account
i talked to you thursday after class about maintaining my user
account at the ai lab, and you said i should send you my name and
user name; here it is
phil wadler [1,plw]
thanks.
∂06-FEB-76 1646 PAW letter to Leon Knopoff
i need an address for Leon knopoff
∂06-FEB-76 1650 PAW documentation
can you, sometime this weekend, finish going through that pile of documentation
on the hall table? Then I can starting getting the files organized and indexed.
Thank you.
}
Math made difficult.
∂06-FEB-76 0946 JMC crunch
Peter Neumann called me back and referred me to Bill Kautz at SRI who
said he would look up a few papers. Let me suggest that you telephone
Bill Kautz on Tuesday to see what he knows both about the techniques
themselves and the references. No-one seems to know about any general
crunching programs so maybe this will be the first utility program
for general use in a time-sharing system -strange as that may seem.
If so, a short article should be written.
[REM -- "First", subject to the condition that it be designed for
ASCII or other text files. Recall that JMCWT1/JMCWC1/PACK/UNPACK were
general-purpose crunching for word-oriented files such as .DMP files.]
True, and maybe we want to revive them together with ? proper writeup.
∂08-FEB-76 2109 KRD an amusing afternoon...
I've discovered I need one more faculty member to be part of my
orals committee, and wondered if you'd mind spending a few hours
some afternoon later this month (the date is tentatively set for
Friday, February 27, 2:15 PM)
I've been working with the MYCIN system (AIM 266 is a good overview
of it); in particular I've been implementing a system which couples
the medical expert directly to the program. The idea is to let the
expert (who knows nothing about programming) `communicate' with the
system, so that he can observe its performance, track down bugs,and
then add (or revise) the knowledge in the program's knowledge base
to fix the problem. The approach used is domain independent, so that
none of the work is limited to medical domains; the interaction is
kept `high level', so that people unfamiliar with computers can still
use the system.
I'll have a draft of the thesis ready in about a week, which would
give you ten days or so to look it over.
Let me know if you might be able to make it. The date can be altered
if the current one is unsuitable.
Randy
∂09-FEB-76 0653 REM GRIMM[LIB,DOC]
It turns out the listing for the time I ran it has been on my
disk area all along. The c.r. is indeed <44%. The listing file
is HH6B.LST[1,REM]. ET .../R if you want to peruse it like we did
the other day with the wrong listing file.
∂09-FEB-76 0722 REM WUTHER[LIB,DOC]
The computer wasn't very busy this morning, so I ran CRU1 on
WUTHER[LIB,DOC] -- C.R. < 47% WITHOUT CODE, 48% WITH CODE.
Listing file is HH9A.LST[1,REM].
∂09-FEB-76 1937 WD National Engine Removal Week
I have talked to Marty about including it on a trip he plans to
take to Berkeley soon, and will know soon if it is ok.
Are you merely bugging me on principle, or is ther something that
you plan to do for which it is in the way?
∂09-FEB-76 2246 100 : REM via AMET Wuther, more left-context, ...
The effect of more left-context will be a smaller encoded text
at the expense of larger code-description, slower-running, larger core
image while encoding and decoding, and more sensitivity to deviant data
in other files being crunched or to the same file after editing. I have
been pushing in the other direction as much as possible this month
because of your desire to have the program run quickly. We can always
vary parameters on an almost continuous basis as estimates of the
memory/time tradeoff costs for our system change. Meanwhile I'm working
on getting the general history cruncher up and running as soon as
possible. After we know how fast it runs, we might decide that it
needs to be speeded up more, or we can afford to slow it a little to
get a better code, we'll see...
OK.
∂10-FEB-76 0640 FTP:MASINTER at PARC-MAXC file understanding system
Date: 10 FEB 1976 0639-PST
From: MASINTER at PARC-MAXC
Subject: file understanding system
To: JMC at SAIL
Cordell Green was talking to me about some ideas you had for a
"file understanding system". I'd like to find out more about what
you had in mind -- I'm looking for a thesis topic.
When would be a convenient time for me to come?
Larry
-------
Friday at 4pm would be fine, otherwise telephone.
∂10-FEB-76 1210 ZM Aharon Gill
I wrote today to Aharon Gill asking him for all the non-classified
information he can give about his project.
∂10-FEB-76 1631 100 : REM Bill Kautz
I talked to him, and he didn't have anything specific...
Once they at SRI had been consulted to work on compression, and had decided
that a 4:1 compression ratio (25% the way I measure it) could be obtsa$ned,
but it was never implemented and nothing was written up, so there is
no way to know if it would have worked or what the various tradeoffs
would have been.
Don Knuth pointed me at Dick Sweet at Xerox who pointed to Ed McCreight
also at Xerox who told me that he had implemented a cruncher that
remembered all repeated strings in the text and outputted a text that
replaced the repetitions of long enough strings by pointers to the
first occurence. The decompressor is easy, but there is a fancy tree
generating technique in the compressor for finding the repeated strings.
According to McCreight it works well for BCPL programs, but not so
well for English texts, but he used smaller English texts. On the
face of it, it would appear that your technique and McCreight's could
be combined. His telephone number is 494-4462, and he has promised
to send me a reprint. I guess I think you should proceed as you are
going for now. There are some IBM leads that Dave Grossman is checking
for me, but that's all.
∂10-FEB-76 1810 REM Crunch, purge, ...
To: RWG, JMC, REM
If the crunching process turns out to be very-fast, then since
tape-dump time is proportional to size of data written on tape rather
than original data, we might crunch-and-dump to make the dump go faster.
This might apply to both purge and normal dump, after our process is fully
reliable.
∂10-FEB-76 1829 REM System support for large use of crunching
To: JMC, REM
There will be totally different (more or less) schemes for crunching
different types of files. For example, for files crunched by the history
method as it now stands, there will be the fact that the history method
is used at all, the polish or tables for left-context, and the polish or
tables for the actual huffman codes used in each context. The first of these
will be fixed for almost all text (ASCII) files, the second will be fixed
for large classes of files (Prose, FAI, SAI, ...) and the third will
probably be computed separately for each file but not changed when the
file is edited. In the retrieval for each file could be
pointers to the default crunch-method etc. for that file, and a flag to
indicate whether the file is currently crunched or plaintext. When the
phantom wants to crunch a file, it looks at this info to quickly load
the appropriate cruncher, unless that info is null (initial condition) in which
case it scans the file to decide which method looks most reasonable.
After trying a method, if it succeeds then the info is set to indicate that
scheme in the future, otherwise the info is set to say "normal crunching
methods fail, don't ever crunch this file until a new method is
developed". -- Does it sound like a good idea?
These ideas seem good, but I would like to begin with a simple
utility program without assuming any system changes. System
changes are very expensive right now, because of other changes
associated with the KL-10.
∂11-FEB-76 0605 REM via AMET SO FAR
CRU2.FAI is now able to parse the polish files that contain
the tree of reversed left-contexes and the huffman-tree at each node,
and compile them into tables to be used for both crunching and uncrunching.
The total runtime for loading the program from a .DMP file, allocating core,
parsing files, compiling tables, is about 1 second. This is fast enough
that we can afford to keep only the polish around and recompile tables
when needed. The tables occupy about 5K (10 pages) of core. After a sleep
I will be ready to write the code that accesses the tables to actually
crunch and uncrunch a file.
∂11-FEB-76 1215 LES ONR Proposal
To: ZM, Waldinger @ SRI-AI
CC: JMC, HVA
Bill Massy signed off on the PI waiver this morning, so the proposal
should get out of the University in the next few days.
Congratulations on getting through the Stanford bureaucracy.
∂12-FEB-76 0328 REM via AMET Working crunch and uncrunch
The code I put in tonight to perform crunch and uncrunch, using the
tables I told you about yesterday, seem to both work. I crunched and
uncrunched PCHECK.WD[UP,DOC] and then compared the resultant file with
the original. SRCCOM said they were identical. The current system is
in a preliminary stage and is hence a little awkward to use, but it is
ready to perform any experiments you want, including
possibly crunching various files using a code developed for other files.
I have not yet written the ultrafast code for uncrunching and the interface
code for entering and leaving it, so our speed figures won't be accurate
yet, but will be in the proper range at least.
∂12-FEB-76 0334 REM via AMET
P.S. BINCOM also shows them identical...
∂12-FEB-76 0543 REM via AMET
With a giant code, see 76212A.LST[1,REM], it takes 4-5 seconds to compile polish into core.
∂12-FEB-76 0551 REM via AMET Using a code to crunch other files not surveyed, first result.
SUDS.RPH[UP,DOC] was not one of the files used to compile the giant code
listed in 76212A.LST[1,REM]. I tried using it as input with that code,
with these results:
SUDS.RPH 26.3 K
CRUNCHED 15.5 K
UNCRUNCH 23.4 K (DIRECTORY PAGE ETC. DELETED)
The giant polish itself is about 1 K, and when compiled into CRU2 the core
image is 52 pages (26 k) altogether (program, buffers, compiled tables for
both crunch and uncrunch).
I have had a few thoughts: First the numbers from WUTHER and GRIMM indicate
that the using a larger tree would produce more compression than
the increased size of tables would lose. So, if you can devise a
fast way to increase the tree, there would be payoff. Second, after
the current version is working it might be worthwhile to experiment
with a word oriented cruncher. Third, I am sending you the McCreight
paper, but it isn't clear to me that it is apropos. Fourth, I have
been thinking about algorithms for finding repetitions of long sequences
in large files, but I haven't come to a definite conclusion yet.
Some experiments may be necessary.
∂12-FEB-76 1946 HPM BLACK WATCH
Oh well. It turns out that there are more differences between
the kits than I thought. Your crystal is bigger, and doesn't fit.
Also (incidentally), your display chip has 4 large (instead of 5 small,
the middle one unused) digits. I'll do the assembly anyway.
∂12-FEB-76 2023 REM via AMET SNEKOT.DAT
I guess I wasn't clear. I didn't mean that I was giving you
permission to edit files on my area, nor that my files were unprotected,
nor that it would be easy for you to do it yourself. What I intended was
that it was trivial for me to do it if you had a list of left-contexts
you thought ought to be added. The program CRU1.FAI is not intended as
a user program currently, and hence no effort has been made to make it
easy to use. If you wanted to copy SNEKOT.DAT to your area and play with
it there, and if I instructed you on the pecularities of CRU1, it might
be possible for you to conduct independent experiments. However, since
I have some information already on some left-contexts that don't help and
may actually hurt, for various reasons, submitting suggested left-contexts
to me and letting me comment on them and/or try them would probably be
most productive. Of course, some live experimenting on your part
might help you understand things more and hence improve our level of
cooperation on the project.
∂12-FEB-76 2049 REM via AMET
CRUCOM.PRO[1,REM] discusses possible command syntaxes etc.
Ok. These are based on some common words. ( him), ( they), ( their)
( there) ( cann). Please add them to the file and try WUTHER
again.
∂13-FEB-76 0035 REM via AMET WUTHER, SNEKOT, CRU1, SYSTEM
(1) The system is alittle too busy now to get any work done, so I'm gong
to wait until later. A few comments regarding SNEKOT.* and CRU1 however,
the contexts you sent me were forewards. To feed them to CRU1 you must
reverse them, and include all shorter histories that are final (initial
in the reversed notation) segments of them. Then you must find the
correct SNEKOT.* file you want to augment and append that file with your
new reversed-strings, then run SSORT to put them in a reasonable sequence.
Finally you must get CRU1 to read in the new file and the text files you
want to survey. It's a good thing I'm going to do it myself this first
time, it could be quite confusing to not know what's wrong when CRU1 does
a HALT AT USER MUMBLE!
(2) If you are satisfied with my work so far and want to have it developed
into a usable program, not only for prose but source programs, not only
the crunch part (like now) but also the spindle, and with acceptable error
messages (in case of error) and an acceptable command syntax, I request
at least a second month of employment. Because I have found it difficult
to get 40 hours a week done (with effort I can just make it) partly because
of the unavailability of the system (due to being too busy) much of the time,
I think 3/4 time would be a better arrangement. -- Employment extension
hereby submitted, awaiting reply...
(3) I'm also considering changing the way <CR><LF> is handled to make it
uniform in all files without preventing bare <CR> or bare <LF> from being
in the file. We will probably want that change before we put it up as a
user program (currently it processes a bare <CR> or a <CR><LF> pair but
halts if a bare <LF> appears in the input because it has no room to put a
129th character).
I am satisfied with your work, and am agreeable to another month at
3/4 time, but I clear all financial commitments with Les who gets
a copy of this.
∂13-FEB-76 1202 REM WUTHER WITH AUGMENTED LEFT CONTEXTS
I have added the things you requested, and have run it on
WUTHER[LIB,DOC] by itself, collecting output in 76213A.LST[1,REM].
To aid you in identifying contexts where the next character ought
to be guessable if more left-context items are added, I have run RANTXT
in the program after that, generating 76213A.GIB[1,REM] as a result.
This is pseudo-random text as generated by the Markov process defined
by the raw counts in each left-context. If you see any totally absurd
sequences of letters, then the last character of them may be guessable
if the n-1 characters preceeding each one is added as a left-context to
be considered in the survey. Note that the 46% c.r. in 76213A.LST
is not comparable to c.r.'s obtained when a large class of files was
scanned the other day.
I need to talk to you; please phone.
Here is a message from Cerf that refers to something you said:
∂13-FEB-76 1239 CRF via ISI NSF DIAL-NET PROPOSAL
To: LES, JMC
LES AND JOHN -
I AM SORRY NOT TO HAVE FULFILLED MY PART OF THE TEXT
GENERATION. THINGS HAVE GOTTEN FAIRLY OUT OF HAND SINCE WE
TALKED IN DECEMBER -- LIKE THE CS AND EE COMPREHENSIVE/QUAL
OCCURRING CONCURRENTLY AND OTHER COMMITTMENTS (LIKE TO ARPA)
HAVE GOTTEN IN THE WAY. I HOPE YOU DID SEND OUT THE PROPOSAL
ANYWAY. MY INFORMATION IS THAT CABLEDATA ASSOCIATES (PAUL BARAN AND
PROF. ED PARKER/COMMUNICATIONS DEPT.) HAVE A PROPOSAL UNDER REVIEW
WHICH PROPOSES TO DESIGN AND BUILD A MESSAGE HANDLING SYSTEM
USING DIALED TELEPHONE LINES AND MICROPROCESSORS. THEIR
IDEA DIFFERS FROM DIAL-NET IN THE SENSE THAT THEY DID NOT PROPOSE
THAT THEIR MESSAGE SYSTEM BE COMPATIBLE WITH THAT OF THE ARPANET.
I DON'T THINK THEY HAD IN MIND THAT THE MICROPROCESSOR WOULD SPEAK TO
HOSTS AS IF THEY WERE HOSTS, BUT RATHER AS SIMPLE TERMINALS. THERE
IS SOME QUESTION ABOUT THE NUMBER AND LEVEL OF PROTOCOLS INVOLVED.
JEFF RUBIN'S CONCERN OVER THE DELAY (NOT THE RELATIVE OVERHEAD) OF
PROTOCOLS ON RELATIVELY LOW SPEED LINES (E.G. 300 BAUD) IS WELL TAKEN.
HOWEVER, TO ACHIEVE ANYTHING LIKE THE RELIABILITY OF THE ARPANET
MESSAGE SYSTEM, IT WILL BE NECESSARY TO INTRODUCE AT LEAST AN SDLC OR
HDLC LIKE LINE CONTROL PROCEDURE JUST TO HANDLE SEQUENCING, DULICATE
DETECTION, AND FLOW CONTROL. ON TOP OF THAT, ONE WOULD EXPECT TO SEE
SOME MINIMAL KIND OF MESSAGE FORMAT, BUT THIS NEEDN'T INVOLVE ANY
SPECIAL CHARACTERS OR PROTOCOL, BUT RATHER MERELY
PRESENT A UNIFORM TEXTUAL APPEARANCE WITH LABELLED FIELDS LIKE
"TO:", "FROM:", "CC:", "SUBJECT:", AND THE LIKE. SDLC AND
HDLC ARE BIT STUFFING PROTOCOLS AND SOME PEOPLE HAVE CHOSEN TO
IMPLEMENT VARIANTS WHICH USE THE OLD ASCII SYN/SYN/DLE/SOH/
<HEADER>/DLE/SOM/<MESSAGE>/DLE/ETX/CRC/CRC/CRC... FORMAT, IN WHICH
THE HEADER CONTAINS INDEPENDENT NUMBERING IN EACH DIRECTIONS SO
AS TO EMULATE THE INTENTION OF SDLC/HDLC WITHOUT BIT
STUFFING.
THERE REMAINS APROBLEM WITH THE INTEGRATION OF SUCH A STRATEGY (LINE CONTROL
PROCEDURE + TEXT FORMAT) WITH EXISTING ARPANET HOST SOFTWARE, SINCE
THE HOST-HOST AND FTP LEVELS WOULD BE MISSING. A POSSIBLE SOLUTION
TO THIS PROBLEM IS TO INTRODUCE A SUBSYSTEM IN ARPANET HOSTS
WHICH IS WAITING FOR CALLS ON KNOWN MODEMS AND WHICH WILL BE PLUGGED
INTO THE MESSAGE SYSTEM AT A LEVEL ABOVE THE FTP; ALMOST LIKE USER
TYPING TO MAIL OR SNDMSG. I WOULD BE INTERESTED IN YOUR REACTION (OR
JEFF'S) TO THIS IDEA AND ALSO YOUR FEELING ABOUT THE MAGNITUDE OF
THE PROBLEM (MAYBE THERE'S NO REASON TO WORRY ABOUT THE DIALNET
TERMINALS TALKING ARPANET STYLE SINCE THEY CAN BE USED TO CALL
UP EXISTING ARPANET MESSAGE SYSTEMS VIA TIPS OR OTHER MEANS OF
ACCESS INTO THE NET, BEHAVING LIKE ORDINARY TERMINALS.
CHEERS.
VINT
∂15-FEB-76 0013 KRD oral exam date, finally
Dear committee:
Thanks to all for the patience with getting this scheduled. The
final agreed upon date is WEDNESDAY, February 25, at 9AM, in the
SERRA HOUSE conference room. A draft of the body of the thesis will
be in your hands by this Tuesday at the latest.
Thanks again.
Randy
∂15-FEB-76 1838 CG WISE MAN PROBLEM
The proof for the Wise Man Problem in complete. The following are proved:
The first wise man does not know the color of his spot;
Neither does the second;
But the third does. The axioms are in KNO.AX[FOL,CG], and a SHOW of
the proof in KNO.PRF[FOL,CG]. I should warn you that the principles on
which the axioms are based are somewhat different from those I discussed
at the seminar.
∂15-FEB-76 2101 REM CRU1 AND CRU2
CRU2 has been modified to include the original filename as part of the header
in the crunched file, thereby making it reasonable for somebody to actually
crunch a file using it. (If later he uncrunches a random crunched-file he
finds on his disk area after forgetting what it was, he will be able to
find out its original file name, which was hopefully meaningful to him.)
I have not yet made the bare-linefeed change.
First timings for CRU1 and CRU2, using NOTICE[UP,DOC] as input file.
CRU1 Scan file 1:39
Type counts, ↑O 0:30
Generate code ↑O 0:39
CRU2 Crunch 2:16
Uncrunch 2:55 (should be sped up when cross-links are done)
all times are in min:sec runtime as typed out by the TI command
∂16-FEB-76 0428 REM TIMING DATA ON CRU2
I have implemented phase 1 in the speedup, partial-compiling of the
shortcuts from one history-output node to the next history node
(top of the output tree). It initially is all zero, i.e. no links,
and each time a character is output (except for ESCAPE, which is variable
and hence cann{t be hard-ired) the link is used to directly find the next
history n{de ithout scanning, unless the link hasn't been compiled yet
in which case it is compiled on the sp{t t{ be used next time the
same circumstances arise. -- This compile-as-y{u-t{${ [EXCUSE THE GARBAGING
DUE TO TTY11 IN ITS USUAL MARGINAL STATE] COMPILE-AS-YOU-GO makes the
program run significantly faster. Runnin$ a sec{nd time after it has
compiled all the links at least once isnrt si$nificantly faster. Here is the
timing, as produced by the TI command. The file is NOTICE[UP,DOC] which
I have chosen as the most typical mess of text {n {ur system.
OLD METHOD -- 2:55'39 2:33r32 2:38r24
WHILE PARTIAL-COMPILING -- 2:05'44 2:02r28
ALREADY FULLY COMPILED SHORTCUTS -- 2:01r23 2:09r03 2:01r45
NOTE THAT ON TTY11, IN THIS MESSAGE, THE FOLLOWING TRANSMUTATIONS OF
CHARACTERS FREQUENTLY OCCUR:
LOWER CASE O TO OPEN-CURLY-BRACE
LOWER CASE W TO NOTHING AT ALL
APOSTROPHE TO LOWER-CASE R
LOWER CASE G TO ALTMODE OR DOLLARSIGN
IF ANYTHING DOESN'T MAKE SENSE, TRY THE INVERSES OF THOSE AND IT MIGHT.
∂16-FEB-76 0845 REM TIMING RESULTS WITH FAST-CODE IMPLEMENTED ALSO
I have now implemented the fast code l{{p, ith the result that
whereas the ori$inal probram t{{k ab{ut 2 min and 40-50 sec{nds,
and the c{mpile-as-it-ent pr{$ram t{{k ab{ut 2 min and 5 sec{nds,
the fast pr{$ram takes {nly ab{ut 45-50 sec{nds.
The file is NOTICE[UP,DOC] which uncrunches from 22.4k t{ 4{.3k
TOO BAD THE DIGIT 7 CANNOT BE TYPED ON TTY11 IN SOME CONTEXES!!!!!!
I'LL TRY AGAIN FIGHTING WITH TTY11...
The file is NOTICE[UP,DOC] which uncrunches from 22.4k to 4{.3k
47.3k
(The original file is 62.0k but much {f that is direct{ry pa$e and
nulls at the end {f each pa$e, hich are flushed by the crunchin$)
∂16-FEB-76 0854 REM FASTER THAN COPY!! MORE POWERFUL THAN A LOCOMOTIVE.
Able to uncrunch steep buildings in a single bound!
For comparison, I tried COPY NOTICE.[UP,DOC]/N and it took {ver 53 sec{nds.
I tried COPY <UNCRUNCHED VERSION OF NOTICE>/N and it took over 47 sec{nds.
Since uncrunching took {nly 45-4{{{{{{{{{|7 sec{nds (fuck tty11)
45-47 sec{nds.
I conclude that uncrunching is faster than c{pyin$$n
copying$$$$$$$$$$$$$$$$
/n
copying/n
Thus it is "fast enough"!!!!
Note that smaller files should run$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$|$$`@////
uncrunch pr{p{rti{nally faster, except f{r the se$$$$$$$$$$$
startup sl{d{n inherent in the c{mpile-as-y{u-${ if the file hits each
node a small number of times (1 {r 2)...
I GIVE UP ON SENDING LOWER CASE MESSAGES ON TTY11!!!!!
NOTE THAT SMALLER FILES SHOULD UNCRUNCH PROPORTIONALLY FASTER, EXCEPT FOR THE
STARTUP SLOWDOWN INHERENT IN THE COMPILE-AS-YOU-GO IF THE FILE IS SO
SMALL THAT EACH NODE IS ONLY HIT ONCE OR TWICE.
SO, NOW THAT CRU2.FAI UNCRUNCHES FILES AS FAST AS A SPEEDING BULLET (COPY),
AND WITH C.R. APPX 50% $N MOST TEXT FILES, LOOKS LIKE WE'RE "COOKING WITH GAS".
Yes, it looks good enough to try on users, so the next point is the
human engineering which I will want to try before inflicting it. I
suggest that the version of the cruncher be stored with each crunched
file so that changes can be made occasionally without requiring everyone
to uncrunch and recrunch. File ages are unreliable if crunched files
are copied.
∂16-FEB-76 1059 BPM MAIL PAT%IMSSS
To: JMC, LES
The people at IMSSS desire to put up a mail protocol now. Besides me, is there
anyone who uses IMSSS regularly or who likes to communicate with the people
there (Suppes, Bob Smith, etc.)? If there is much demand besides myself, I think
it would prove very useful.
Sure it will prove useful, and I am happy that you are volunteering.
∂16-FEB-76 1538 REM VERSION-NUMBER IDENT
Have no fear!! Already in JMCWT1/JMCWC1 back there in 1971 I had already
begun using secret passwords for crunched-file identifiers. The code is
a 16-bit number that represents the year/month/date that the system was
designed, approximately of course, so it is impossible for it to repeat until
one of my systems lasts 128 years and I happen to design a new one on the
same day of the same month so far away. I have added one new ident to
make things even safer and more understandable. In the JMCWT1/JMCWC1
system, the first 16 bits of the file were that password. In all the new
systems I have made up this year, the first word consists of the ASCII
string REM<CR><LF> so that users know who to ask for help, and the next
16 bits after that are the secret password. If a program trys to uncrunch
a file whose password is unknown, at the very least it can type out the
16 bit number and ask the user to ask REM for help, or it can swap to
most of the 16 bit passwords, which can [BUG IN MAIL -- THIS LINE IS EATEN AWAY]
∂16-FEB-76 1557 REM CONTINUATION OF LAST MESSAGE...
... so the master program could know all the passwords that have
been used since 1971 and direct the user to the correct uncruncher one
way or another.
Regarding user interface. One way to work that out is for us to
send messages back and forth, your messages represent things the user
types in, my messages represent program output. I could be actually
running my program at this end, RAID and all, to get a true effect,
or I could be just pretending. Alternately I could try to guess the
input and prepare a sample teletype transcript for you to examine and
criticize.
I suggest you prepare a few sample dialogs.
∂17-FEB-76 0224 REM
CRUCOM.PLN[1,REM] contains some preliminary comments and a sample dialog.
∂17-FEB-76 0759 REM via AMET
CRU2 can now write out the polish as part o the file. Still needs parser.
∂17-FEB-76 1409 FXB 11/70
I give up. Why is an 11/70 like a 200 pound canary?
∂17-FEB-76 1814 HQM imlac graphics
oh well, it looks like the skip in stanford imlacs
has been modified in some obscure manner.
Sincetheres no way to get the source files of
the nice programs for the machine, I guesss
its a losing battle.
HQM
As you may have guessed, I view this with equanimity.
∂18-FEB-76 0106 CG WISE MAN PROBLEM
I have just finished some documentation on the wise man problem proof.
KNO.NEW[1,CG] is a rough rewrite of KNO, with the new ideas on which
the proof is based included. (Only page 3 has been changed). KNO.DOC[FOL,CG]
contains a commented version of the FOL axioms for the proof.
∂18-FEB-76 0212 DCL Gerard Huet summer visit
Huet's summer visit could come out of savings on my
salary while at Harvard.
suggestions:
one month...per diem consulting, total cost about $1000
three moths...regular ress assoc. salary $1400 per month.
I'm not sure how long he wants to come for and will cable
to ask him after you reply to this.
-David
∂18-FEB-76 0905 REM
A crude version with user-level scanner now exists, CRU2.DMP[1,REM]
It will remain crude until the user-level syntax is decided upon.
I haven't received any feedback fromyou yet re$ardin$ CRUCOM.PRO
crucom.pro looks ok, except that "crunch foo" should have the
effect of "crunch foo←foo". I am asking reg for an opinion.
SPINDL will pose more problems.
∂19-FEB-76 0124 REM IMPROVED CRU2.DMP[1,REM]
You might try it to see h{ it c{mpares ith hat is described i ...xxx
YOU MIGHT TRY IT TO SEE HOW IT COMPARES WITH WHAT IS DESCRIBED IN
CRUCOM.PRO -- TELL ME WHICH PARTS YOU LIKE BETTER THAN CRUCOM, AND
WHICH PARTS WORSE. THE EASIEST WAY IS TO RUN IT THROUGH PTYJOB AND
THEN EDIT THE TRANSCRIPTION TO CONTAIN YOUR COMMENTS, THEN MAIL REM
A NOTE TELLING THE NAME OF YOUR TRANSCRIPTION/COMMENT FILE. THE TOPLEVEL
PACK.DMP AND UNPACK.DMP ARE, OF COURSE, NOT PART OF THE CURRENTLY-IMPLEMENTED
SYSTEM, AND CRU2 DOESN'T YET READ TMPCOR FILES.
THERE ARE SOME HISTORY AND HUFFMAN TREE FILES ON MY AREA,
FILE NAME HIST.* AND HUFFS.* -- BE SURE TO USE A MATCHING PAIR, OR YOU'LL
GET A HALT IN THE MIDDLE OF THINGS.
I don't know about running things through ptyjobs - never done it. Is
it described in monitor manual. I will not have time to do much until
next Tuesday, because I am getting a paper finished for the AAAS meeting
in Boston. Delayed reaction: I think the initial system should have
different programs for crunching different kinds of files and the user
should choose the program. Such a system will be simpler to maintain.
I will be here for only another few minutes.
∂19-FEB-76 0136 REM PTYJOB
R PTYJOB
<META>D
OUTPUT FILE = <TYPE NAME OF LISTING FILE>
<DO YOUR THING, RUN CRU2[1,REM] OR WHATEVER>
<CTRL><META>D
Closing output file...
OK, thanks. By the way, I mentioned your garbaging problem to Panofsky
who said he hadn't heard of it, made a few tests and speculated that
your terminal was misbehaving. I suggest you send him a message with
evidence of malfunction.
∂19-FEB-76 0142 REM TTY11 PERSISTANT TROUBLE
To: TED
CC: JMC
A few months ago I told TAG about a problem I had been having with TTY11
garbaging what I type at it, particularily the l{er case characters
"o" "w" "$" (see hat I mean, that sh{uls have been a l{er case tEE//
GEE) "$" "$" "$" "$" and the {ther characters "$" (slant) "$" (questi{n
mark) "{" (seven) and als{ rub{ut. He lo{ked int{ it, put the sc{pe {n
the input, and determined that my terminal as emittin$ prefect ASCII
as far as he could measure. He tried replacing some boards, hich didnrt
help, and [INTERRUPT, "'" ALSO GETS MUNGED] since nobody was having
the problem he decided it was s{me c{mbined mar$inality beteen the terminal
I have the the m{dem$$$ datasetas$$$ datasets y{u have there, and decided
not t{ {rk {n it further. As y{u seein this messa$e, the $arba$in$ is
quite bad. This pr{blem has never happened {n any {ther lines, TTY10,
AMES-TIP, IMSSS-DIALUP, SRI-AI-DIALUP, so I know the fault doesn't lie
entirely in my terminal, even if it may be sli$htly mar$inal.
JMC suggested I tell yo avou$$$ ab{ut it.
∂19-FEB-76 0545 REM via AMET
P.S. *.NOT[1,REM] (Derived from NOTICE[UP,DOC]) are reccommended for crunching.
∂19-FEB-76 0600 REM via AMET IMPROVEMENTS TO CRU2
There is a check to be sure the number of nodes in the history tree plus 1
equals the number of trunks in the Huffman-trees file, whenever the latter
is loaded. When a file doesn't exist, or a output file already exists,
or a password in a file is unknown, it now types reasonable messages and
returns to command level. Program tells c.r. (appx) when crunching, and
exactly when uncrunching. (In the former case the possibility of line-numbers,
directory-pages, nulls etc. cause ambiguity in definition of c.r.)
If a history tree is loaded and one is already loaded, core used for both
the old history tree and Huffman trees are recycled. If a Huffman tree file
is loaded, just the old Huffman trees are recycled. Thus except for the
command scanner (it asks you one thing at a time, rather than accepting
a full command COM OFILE←IFILE/SWITCHES<CR>) it seems to be in usable shape.
Of course, some system programs, such as COLIST, operate in one-at-a-time
mode, so I guess CRU2 is acceptable now, modulo any bugs I haven't found yet.
So, CRU2[1,REM] awaits your trial...
∂19-FEB-76 1205 REG
∂18-FEB-76 1529 JMC
The correct English word is "IMPURE".
1. My dictionary lists "Unpure"
2. The UUO really "unpurifies" a segment. ("impurifies" is not an English word)
I stand corrected
∂20-FEB-76 0458 REM via AMET New features in CRU2
CRU2 now includes a hash number for the history-tree and a hash
number for the Huffman-trees in each crunched file. Thus if you don't
include the trees in the crunched file, the program can at least check
at the onset of uncrunching to see if you are going to win.
There was a bug in the Yes/No routine, it took all answers to
be Yes regardlss of what you typed.
Unless more bugs exist, it is ready for you to play with.
Unfortunately, I am going to Boston tomorrow, have just finished my
paper and won't be able to get back to it till next week.
∂20-FEB-76 0505 REM at TTY122 0505 via AMET
By the way, besides AAAS and HOTER.ESS [W76,JMC] any other writeups
of yours regarding information retrieval / access etc.?
∂20-FEB-76 1109 FTP:PHW at MIT-AI
Date: 20 FEB 1976 1406-EST
From: PHW at MIT-AI
To: jmc at SU-AI
A copy of the book is on the way. I had planned to
send a more debugged version latter, but maybe that
will happen before you get around to reading this one
anyway.
Yours,
Patrick
-------
∂20-FEB-76 1430 REM
I believe it was referring to default history tree file missing from your area.
I'll change the message to tell you what type of file is missing.
......OESN'T EXPLAIN THE ILL MEM REF OR TELL ME WHAT TO DO NEXT.
i'm here.... I'll be over...
∂20-FEB-76 1721 FTP:Dehall at SRI-AI Berkeley Workshop on Data Management and Computer Networks.
Date: 20 FEB 1976 1718-PST
From: Dehall at SRI-AI
Subject: Berkeley Workshop on Data Management and Computer Networks.
To: Invited Speakers:
cc: DAustin at MULTICS, DEHall at MULTICS
LBL proposes to host a workshop on computer networking and data
base management on May 25-26 1976. Registration will begin May 24
at 7:00 PM.
GOALS:
To provide a summary of the latest techniques in computer
networking and data base management.
To provide a ground for discussion of data base sharing in
a distributed network environment.
FORMAT:
Each session will consist of a short talk by each of three
experts, followed by a discussion period. Sessions
will be in parallel on the 25th, and combined on the 26th.
TOPICS:
1. Implementation and future improvements of existing computer
networks (e.g., CTRNET, ARPANET,EDUCOM,INFONET,MERIT,TELENET).
2. High-bandwidth data-links (e.g, Fermilab-Argonne, LBL-SLAC)
3. Gateways.
4. High-level general and special purpose protocols.
5. Teleconferencing, mail facilities, and network graphics.
6. Shared data bases, both centralized and distributed.
7. Data exchange techniques and issues (data reconfiguration services,
standard interchange formats, security/privacy).
8. Machine-independent data structures and query languages.
9. Data management machines.
PUBLICATION:
The workshop should be considered an informal meeting for
open discussion of current work and future plans in the above
topics. A transcript of the sessions will not be published.
However, invited speakers should contribute short position or
idea papers for the Workshop Proceedings.
Please let us know if you will be able to participate, and
which topic(s) you wish to discuss. We would like to have an
overall schedule ready for distribution by March 5, 1976, so
that invitations can be issued to workshop attendees by March 15,
1976. Suggestions for additional topics and/or speakers are
welcome.
Don Austin Workshop Chairman DAustin@Multics
415-843-2740 Ext 5313 (FTS 451-5313)
Dennis Hall Co-chairman DEHall@Multics
415-843-2740 Ext 6053 (FTS 451-6053)
David Richards Co-chairman
415-843-2740 Ext 5766 (FTS 451-5766)
Computer Science and Applied Mathematics Department
Lawrence Berkeley Laboratory
University of California
Berkeley, Ca94720
Preliminary list of invited speakers:
Herbert Baskin UCB
Harvey Baumel TELENET
Al Brooks ORNL
Jerry Burchfiel BBN
Jed Donnelley LLL
Victor Elischer LBL
Gerald Estrin UCLA
David Farber UC Irvine
Jake Feinler SRI
Robert Fink LBL
Ed Franceschini NYU
Terry Gray UCLA
Fred Gey LBL
Harvard Holmes LBL
John Killeen LLL
Leonard Kleinrock UCLA
Patrick Mantey IBM
James Martin IBM
John McCarthy Stanford
Robert Millstein Mass Comp Assoc
Mike O'Mally UCB
John Postel SRI
Milton Rose ERDA
Arie Shoshani SDC
Michael Stonebraker UCB
Robert Thomas BBN
Jacques Vallee Institute for the Future
James White SRI-ARC
-------
Glad to accept invitation to Data Base workshop.
∂20-FEB-76 1416 JMC BUG
While logged in as ESS,JMC, I had the dialog:
ru cru2[1,rem]
COMMAND ..etc.
Crunch
INPUT FILE NAME
hoter.ess[w76,jmc]
OUTPUT (CRUNCHED FILE NAME:
foo
INCLUDED HISTORY TREE?Y
INCLUDED HUFFMAN TREE?y
(NO SUCH FILE) (NEED TO LOAD HISTORY TREE FIRST) (E/TV FILE INIT'D OK)
Ill mem ref at user 15242
↑C
[REM -- This bug has now been fixed. The problem was that after failing to
lookup the default history file (called from inside the CRUNCH routine)
it said "no such file" and set the error flag, which CRUNCH didn't check.
Next, POLHUF was called from CRUNCH, which noticed that the history hadn't
been loaded, so it said "need to load history tree first" and set the error
flag (a no-op since it was already set) and returned. CRUNCH then again failed
to check the error flag and began processing the file. The first thing it
did was to call HISPOL to write out the nonexistant history tree into
the crunched file, and the code for this routine got the ill mem ref. --
All the above messages have been changed to make them more meningful, in the
event the history tree is missing, and CRUNCH has been changed to check
the error flag and abort itself if either POLHIS or POLHUF failed. --
Also, default history and huffman files have been set up, and CRU2 nows
about them, so the failure conditin shouldn't arise in the first place.]
∂21-FEB-76 1635 REM via AMET Improvements in CRU2
Compile-as-it-runs shortcuts in the history are now in effect for
CRUNCHing (previously only for UNCRUNching). Crunch now takes about
30% less time than previously. Using a smaller tree of history makes it
run faster because after finding the correct history node it enters a
linear search for all the ASCII characters that have their own codes,
(the nodes are sorted, commonest first, but for large lists the average is
still quite slow). I may decide to use the single-file survey that scanned
NOTICE[UP,DOC] instead of the seven-file survey, or make a new seven-file
survey with larger cutoff, for the default code.
∂22-FEB-76 1225 REG
What do you want me to do about REM's cruncher or cruncher document?
∂22-FEB-76 1514 100 : ED FREDKIN via AI LOGGING IN TO SU-AI
Dear John,
I still can't log in, it evidently doesn't like
network useres,(I can't get to the point where
it asks for my password.
Also, what is my project?
One last thing, can you send me
some SU-AI general hints to the user?
Thanks.... ed
now my problem is that i
don't know
how to make this terminal
send you the control z instead
of doing you know what with it,
but I will try
∂23-FEB-76 1552 LES Jack Buchanan call
Jack called and wants to talk to you about your trip there.
∂23-FEB-76 1622 LES
∂23-FEB-76 1040 FTP:MOORE at MIT-AI
Date: 23 FEB 1976 1335-EST
From: MOORE at MIT-AI
To: les at SU-AI
I am very dissappointed by the financial arrangements. While I am not paying MIT
tuition this semester or this summer, I will have to pay tuition while I am actually
writing my thesis, even though I am at Stanford. Plus my moving expenses are over
$2000, and I was counting on the money I saved by not paying tuition this term to
offset these costs. When John offered me support, I assumed it would be similar to
MIT, about $775/mo. Substantially less than this will cause me real problems.
-------
∂23-FEB-76 1802 EF via AI login
john,
Now that I know that "1" is a good
project name, I can log in.
It seems that the problem is mainly the M.I.T. tuition. M.I.T. and
Stanford handle the tuition problem for research assistants
differently, but the first order result is the same. Stanford gives
its RA's free tuition without charging it directly to contracts, but
it is quite unlikely that Stanford can be persuaded to pay M.I.T.
tuition. It seems to me that the solution is for M.I.T. to waive its
tuition while you are here.
∂24-FEB-76 0049 LES Research Assistantship
To: Moore @ MIT-AI
CC: JMC, phw @ MIT-AI
It is my understanding that you need not pay tuition to MIT while you
are here, but that you would pay tuition if you were there.
Therefore, our tentative offer of $450/academic month and $900/summer
month is effectively more than you would receive at MIT. If you have
some conflicting guidance, then please cite your sources.
∂24-FEB-76 0157 KRD reminder
Dear committee:
Tomorrow (Wednesday) at 9 AM in Serra House conference room: my oral
exam. If there are any major questions or severe misgivings that have
arisen out of what you've read so far, I'd appreciate hearing about
them some time today. You can send a msg to me here at SAIL, and I'll
be sure to work the answer into the presentation.
Thanks
Randy
∂24-FEB-76 0939 TED tty11
To: REM, JMC
To: REM
CC: JMC
You are the only one having trouble with TTY11 and I have been unable to
duplicate the problem. I strongly suspect the problem lies with your
terminal. If you will bring it in some day (call first to make sure
I am here) I will look at it. It is very difficult to debug a problem
that is not happening.
∂24-FEB-76 1006 FTP:MOORE at MIT-AI
Date: 24 FEB 1976 1153-EST
From: MOORE at MIT-AI
To: jmc at SU-AI
les@su-ai
I am not enrolled at MIT this term, so the problem is not immediate.
The long range problem is that MIT is very inflexible about paying tuition,
and I will have to pay a full term tuition (about $2000) the semester I
turn in my thesis. The MIT administration has been giving us a lot of
trouble about this arrangement, so I strongly request that they not be
involved. Since the real problem is some months in the future, I will
tentatively accept your tentative offer, and perhaps we can work something
out when I get there. Perhaps the MIT AI Lab could provide some support
that one term. Anyway, today is my last day at the Lab here, so I will talk
to you about this when I get to Stanford in about three weeks.
-------
Perhaps you should spend a final term as an M.I.T. research assistant.
∂24-FEB-76 1400 JMC*
Push for Manna
∂25-FEB-76 0925 LES III licensing deal
I asked John Poitras about the state if III negotiations. He said that
he had reached Fenaughty's secretary on Feb. 13 and explained his interests,
but that Fenaughty was about to jump on an airplane. Fenaughty didn't call
back.
Poitras said that he will try again today.
∂25-FEB-76 1539 RWW ACCOMPLISHMENTS
I PUT SOME THINGS ON THAT FILE. YOU MIGHT LIKE TO READ IT.
∂25-FEB-76 1913 REM via AMET SPINDLE FORMAT ETC.
To: LES, JMC
At last I have a better idea for how to do the spindles. In brief,
each crunched file will be stored without its history and Huffman trees,
but one copy of each different tree-file will be in the spindle by itself.
The crunched files will have in their directory the hash numbers for the
two trees needed for uncrunching, and the directory item for each tree-file
will contain the hash number, so that the appropriate trees can be
selected during uncrunching. Details are in a temporary file at the moment,
but I'll move it to CRUNCH.PLN sometime soon and edit it a little.
This new way to do spindles will probably affect command syntax somewhat.
∂26-FEB-76 0837 LES KL10 delivery
To: JMC, TED, REG, JBR
Just talked to Stafford who said he was told our KL10 will be shipped
on March 26. He is awaiting TWX confirmation and will give it to us
in writing as soon as he gets it.
∂26-FEB-76 1557 JMC Alleged manuscript
So far as I know, no manuscript is due from me on March 1.
I have been demoted from speaker to critic, and it would be
improper for me to write the criticism before receiving
the papers to criticize, although it is sometimes done. Now
that I think about it, you are supposed to send me manuscripts
to criticize. I'll give you until March 2. Seriously, if an
Daniel Bell has sent in a manuscript, I'll be able to discuss it
better on March 9 or 10 as the case may be if I can get a copy
in advance. I am leaving fo
he East on March 4, so it would
be good to have it by then.
John McCarthy